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EXTENDED DEPTH OF FIELD IMAGING 
FOR HIGH SPEED OB JECT ANALYSIS 

RELATED APPLICATIONS 

5 

This application is based on a prior provisional application, 
Ser. No. 60/748,888, filed on Dec. 9, 2005, the benefit of the 
filing date of which is hereby claimed under 35 U.S.C. §119 

(e). 

to 

GOVERNMENT RIGHTS 

This invention was funded at least in part with grants (No. 

9 R44 CA01 798-02 and 1 R43 GM58956-01) from the 
National Institutes of Health (NIH) and a contract 
(NNA05CR09C) from the National Aeronautics and Space 
Administration (NASA), and the U.S. government may have 
certain rights in this invention. 

BACKGROUND 

Conventional imaging systems are challenged to provide 
adequate low-light, high-resolution imaging. Objective com- 
ponents used in high-resolution imaging systems need to have 25 
very high numeric aperture (NA) values. Unfortunately, a 
high NA value of the objective component results in a very 
small depth of field in which to view target objects. A small 
depth of field raises significant challenges in achieving and 
maintaining focus of target objects to be viewed during low- 30 
light, high-resolution imaging. If focus of a target object is not 
achieved and maintained, the resultant defocused image of 
the target object at a detector is spread over an unacceptably 
large area of the detector, with a loss in spatial resolution and 
a decrease in the signal-to-noise ratio associated with the 35 
image of the target object. 

Confocal microscopy provides the ability to image cross 
sections of a cell (“optical sectioning”) for the purpose of 
generating a three-dimensional map of cellular structures, or 
to synthesize a single two-dimensional image in which all 40 
cellular structures are in focus. These capabilities are desir- 
able for a wide range of cell analysis applications, including 
co-localization studies, quantifying the translocation of mol- 
ecules between cellular compartments, and the enumeration 
of fluorescence in situ hybridization probes randomly located 45 
in a nucleus . Although confocal microscopy provides a highly 
detailed view of the cell, the repeated scanning required sig- 
nificantly reduces image acquisition rates, and can in some 
cases, induce photo -bleaching of fluorescent probes. 

Currently confocal microscopy is limited by the length of 50 
time required to capture imagery, the types of signals that can 
be collected simultaneously, and the limitation that the cells 
be immobilized on a solid support. The relatively slow speed 
of confocal microscopy can be a limiting factor for many 
applications. Commonly- studied cellular phenomena, 55 

including signaling, internalization of surface-bound factors, 
chromosomal defects, and various morphological transfor- 
mations, can be subject to high cell-to-cell variation, occur 
over a wide and continuous range of values, or occur at low 
frequencies within a heterogeneous mixture of cells. There- 60 
fore, the study of such phenomena can require the observation 
and analysis of thousands of cells, and the application of 
statistical analysis in order to reach robust and repeatable 
scientific conclusions. In such cases, it is often impractical to 
employ confocal microscopy, due to the low throughput of the 65 
technique, despite the wealth of information it can provide for 
each cell. 
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In the alternative, conventional fluorescence imaging is 
generally much faster than confocal image stacking and can 
provide good spatial resolution and fluorescence sensitivity, 
when employing high NA objectives. However, conventional 
fluorescence microscopy is subject to a tradeoff between NA 
and depth of field. As the NA is increased to improve light 
collection and increase spatial resolution, the depth of field is 
reduced by the square of the NA change. Therefore, images of 
weakly fluorescent signals and cellular structures located out- 
side the ideal plane of focus can be compromised. This effect 
is most readily observed in experiments employing Fluores- 
cence In Situ Hybridization (FISH) probes that are typically 
under one micron in size and are comprised of a limited 
number of fluorescent molecules, which can be distributed 
throughout the nucleus or cytoplasm of a cell. A slight defo- 
cus may preclude the detection of dim probes, or cause mul- 
tiple probes located in close proximity to blur into each other. 
Larger amounts of defocus can cause substantial blur, render- 
ing a FISH spot unrecognizable in an image. These tradeoffs 
for increased speed over the highly focused imagery pro- 
duced by confocal image stacking are generally not accept- 
able, given that conventional microscopy, even in automated 
form, is still slow compared to flow cytometry. As a result, 
many studies of cellular phenomena employ both flow cytom- 
etry (for the high throughput study of large cell populations) 
and confocal microscopy (for the detailed imaging of selected 
individual cells). 

The ImageStream™ flow imaging system was developed 
in part to address the gap between the slow, but detailed 
information obtained by confocal microscopy and the fast, 
but limited cellular information gleaned by flow cytometry. 
The ImageStream™ system collects six simultaneous multi- 
mode images (brightfield, darkfield, and up to four different 
fluorescence colors) from cells in flow. High fluorescence 
sensitivity and resolution are achieved by using 0.75 NA 
optics and a 0.5 micron pixel size. 

Several attempts have been made to extend the depth of 
field of such a flow imaging system. For example, U.S. Pat. 
No. 6,583,865 (the disclosure and drawings of which are 
hereby specifically incorporated herein by reference) 
describes the use of a flow imaging system having a tilted 
detector (or a sample flow path that is tilted relative to the 
detector) that effectively increases the depth of field for a 
more accurate enumeration of structures and probes within a 
cell. The technique can be used in connection with a pulsed 
light source to produce multiple images of a moving object at 
different focal planes, or it can employ a continuous light 
source to produce a single composite image incorporating 
information from the object at multiple focal planes. The 
pulsed light source variant is limited in fluorescence sensitiv- 
ity because each image has a relatively short signal integra- 
tion time. The continuous light source variant is limited in 
image quality because the composite image contains both 
in-focus and out-of-focus information at every location in the 
cell. Hence, there is a need for a high speed imaging system 
having an extended depth of field as well as both high fluo- 
rescence sensitivity and excellent image quality. 

U.S. Pat. No. 7,009,651 (the disclosure and drawings of 
which are hereby also specifically incorporated herein by 
reference) describes a flow imaging system in which light 
from an object is split into a plurality of optical paths, and one 
or more of the optical paths are defocused relative to the 
default focal plane of the system, to similarly increase the 
depth of field. U.S. Pat. No. 6,211,955 (the disclosure and 
drawings of which are hereby also specifically incorporated 
herein by reference) describes the use of a stereoscopic imag- 
ing apparatus to view cells from multiple angles, for the 
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reconstruction of a three-dimensional (3-D) map of the cell 
and accurate enumeration of FISH spots in images. The effec- 
tiveness of this technique is limited by the depth of field that 
can be achieved with the imaging system. If the depth of field 
of each detector is less than the depth of the cell, or at least, of 
the nucleus, the spatial resolution of the three-dimensional 
map produced by the technique will vary across the cell, and 
neighboring FISH spots in the image will blur into each other 
and be unresolved. 

While the ImageStream™ flow imaging system represents 
a significant advance over conventional flow cytometry and 
standard microscopy, demanding applications, such as the 
quantization of FISH probed cells, require imaging capabili- 
ties closer to those achieved by confocal image stacking. 

It would therefore be desirable to develop a flow imaging 
system suitable for high-resolution imagery (0.75 NA and 0.5 
micron pixel size), which also exhibits an extended depth of 
field. 

SUMMARY 

The concepts disclosed herein enable the depth of field of 
an imaging system to be increased. Such techniques are par- 
ticularly well suited for enabling flow imaging systems suit- 
able for high-resolution imagery (0.75 NA and 0.5 micron 
pixel size) to achieve extended depth of field cellular images 
similar to those obtained using confocal image stacking. 
Because flow imaging systems can acquire image data much 
more rapidly than confocal microscopy, these techniques will 
facilitate the analysis of large cellular populations. The con- 
cepts disclosed herein further encompass imaging systems 
configured to achieve such extended depth of field imaging. 

If the point spread function (PSF) of an imaging system is 
well -characterized, the known PSF can be used to improve 
the spatial resolution of imagery acquired with the imaging 
system by mathematically de-convolving the PSF from the 
imagery. In the case where object being imaged lies entirely 
within the focal plane, only a single image of the object need 
be acquired. If the object being imaged is extended in the Z 
axis, multiple images of the object must be acquired in dif- 
ferent focal planes order to produce the resolution enhance- 
ment, due to uncertainty about the focal plane of origin of any 
given feature within a single image of an extended object. 
However, a single image of an extended object can be com- 
bined with PSF de-convolution to enhance focus quality 
(rather than resolution) if the PSF is intentionally modified 
such that it is invariant to focal position. The techniques 
disclosed herein are therefore based on manipulating an 
imaging system such that a point spread function (PSF) of the 
imaging system is substantially invariant over an extended 
depth of field. For example, where an unmodified high-reso- 
lution imaging system might exhibit a depth of field of about 
1 micron, a modified version of the same imaging system 
might be characterized as having a PSF that is substantially 
invariant across a depth of field of about 10 microns. Such a 
substantially invariant PSF enables the imaging system to 
integrate light from different focal positions in object space, 
making the modified imaging system relatively insensitive to 
defocus. This property, in turn, enables de-convolution of the 
PSF to remove the spatial broadening and contrast loss inher- 
ent in the unprocessed image, thereby increasing image fidel- 
ity and creating an “in-focus” projected image of the entire 
cell. The concepts presented herein combine the above tech- 
nique for extending depth of field with a means for greatly 
increasing detection sensitivity. The increased sensitivity is 
important to the practice of extended depth of field imaging, 
because the PSF modification tends to blur optical signals in 
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the unprocessed imagery, thereby decreasing the signal to 
noise ratio. Further, the de-convolution process itself tends to 
amplify noise, reducing the effective signal to noise ratio in 
the resultant extended depth of field imagery, so increasing 
5 the signal intensity relative to the noise is a key feature of the 
present invention 

A key aspect of the concepts presented in the examples 
discussed herein is that a wave front of light from the object is 
deformed, such that light from different focal positions is 
to collected. As long as the deformation process is well under- 
stood, processing of the imaging data collected from the 
deformed light can correct for errors introduced into the 
image data by the deformation process, while enabling the 
information corresponding to the different focal positions to 
15 be retained. Thus, after such corrective processing is applied, 
an image with an extended depth of field is obtained. 

Thus, the following steps can be considered to be an over- 
view of an exemplary process disclosed herein: providing an 
imaging system having a substantially invariant PSF (or 
20 modifying an imaging system to achieve a substantially 
invariant PSF), collecting image data from an object, and 
processing that image data to achieve an extended depth of 
field image. De-convolving the image (taking into account the 
modified PSF) enhances image contrast and reduces spatial 
25 broadening, thereby improving image quality. 

The concepts disclosed herein encompass several different 
exemplary techniques for providing the substantially invari- 
ant PSF and the deformed wave front. As noted above, U.S. 
Pat. No. 6,583,865 describes a flow imaging system having a 
30 tilted image plane (either the detector being tilted or the flow 
path of the object relative to the detector is tilted). Several 
improvements to that configuration are disclosed herein, 
including the use of a flow cell having a tilted flow path. 
Significantly, such an approach does not simultaneously col- 
35 lect data from a plurality of different focal positions. Instead, 
as the object moves relative to the tilted image plane, the focal 
point of the imaging system moves to different focal planes in 
the object. A detector synchronized to the motion of the obj ect 
must be employed (i.e., a time delay integration (TDI) detec - 
40 tor), such that images of the object obtained at different posi- 
tions (and at different times) are combined to achieve an 
extended depth of field image of the object. Rather than using 
such a tilted image plane, an optical element configured to 
deform the optical wave front of light from the image can be 
45 introduced into the imaging system between the object and 
the detector. One advantage to using an optical element to 
deform the optical wave front is that light is simultaneously 
collected from an EDF in the object. Thus, a synchronized 
detector is not required (although it may still be desirable to 
50 employ such a detector). Another advantage to using an opti- 
cal element to deform the optical wave front is that the ele- 
ment may be conveniently inserted into or removed from the 
optical system. Different imaging applications may require 
more or less depth of field, and having a removable element 
55 allows the depth of field to be tailored to the different appli- 
cations of the imaging system. A phase plate (an exemplary 
phase plate can be obtained from CDM Optics of Boulder 
Colo., marketed as a Wavefront Coded™ element) represents 
one type of optical element that can be used to deform the 
60 optical wave front. Yet another type of optical element will 
deform the wave front by introducing a spherical aberration 
into light from the object. A separate optical element (such as 
a cover slip) can be used to introduce spherical aberration, or 
an existing element in the flow imaging system (such as a flow 
65 cell or cuvette, or an objective lens with a correction collar) 
can be modified to introduce the spherical aberration. Where 
the optical element is a phase plate or wave front coded 
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(WFC) element, such an optical element will be disposed in 
infinite space, otherwise known as aperture space (i.e., behind 
the objective lens). If the optical element introduces spherical 
aberration, such aberration is preferably induced before light 
is collected by the aperture of the imaging system (i.e., 
between the object or cell being imaged and the objective 
lens). Essentially, the unmodified imaging system (i.e., the 
imaging system without the distortion element) is configured 
to acquire an image of the object with a relatively narrow 
depth of field (for example, about 1 micron, understanding 
that such a value is intended to be exemplary, and not limit- 
ing). When the distortion element is introduced into the imag- 
ing system, the distortion element induces defocus in the light 
from the object, such that the relatively narrow depth of field 
is expanded (in an exemplary, but not limiting embodiment, 
the defocus extends about + 1-1 microns beyond the original 
depth of field); however, such defocus “blurs” the extended 
depth of field (such blur generally includes both spatial 
broadening and a loss of contrast). Post image acquisition 
processing can minimize the blurring effects of the defocus 
induced by the distortion element, enabling an EDF image to 
be generated. Note that the PSF of the imaging system with 
the distortion element in place is used to facilitate the post 
image acquisition processing employed to reduce the effects 
of the defocus induced by the distortion element. 

Another aspect of the concepts encompassed herein is 
directed to a flow imaging system configured to provide the 
above-described extended depth of field images. Such a flow 
imaging system will include an optical element to deform the 
wave front of light from the object while providing a substan- 
tially invariant PSF, a collection element to direct light from 
the object along an light path, an imaging lens configured to 
form an image from the collected light, a detector configured 
to detect the image and generate image data, and a processor 
configured to process the image data (i.e., to de-convolve the 
image data based on the invariant PSF) to produce an 
extended depth of field image of an object. In some exemplary 
embodiments, the imaging system includes a dispersion ele- 
ment that disperses the collected light before imaging, and in 
some exemplary embodiments, the detector is a TDI detector, 
configured to output image data based on a plurality of images 
of the object detected over time. 

Referring to the ImageStream™ system noted above, the 
concepts encompassed herein can be applied to the Imag- 
eStream™ system, to enable extended depth of field imaging 
to be achieved. An ImageStream™ system modified for 
extended depth of field (EDF) image collection can provide 
for the collection of high-resolution imagery (0.75 NA and 
0.5 micron pixel size) without the defocus associated with 
high NA optics. Such imagery will have a greatly extended 
depth of field (a proposed EDF ImageStream™ will achieve 
a depth of field of ten microns, which is approximately five 
times the current operating single focal plane depth of less 
than two microns), allowing for all image features within a 1 0 
micron EDF to be clearly in focus. This technology will 
enable imaging of cellular components having fine structures 
that are in different focal planes (e.g., cytoplasmic proteins, 
such as actin, microtubules, and sub-cellular organelles (such 
as mitochondria), cellular micro-domains (e.g., membrane 
caps, lipid rafts, protein co -localization, and signal transduc- 
tion), and fluorescent in-situ hybridization spot counting. Sig- 
nificantly, post-processing of the imagery minimizes the 
effects of defocus by bringing the entire cell into focus at the 
same time. Unlike confocal image stacking techniques, this 
new methodology and apparatus will operate at hundreds of 
cells per second, allowing tens of thousands of cell images to 
be collected for quantitative analysis in several minutes. 
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This Summary has been provided to introduce a few con- 
cepts in a simplified form that are further described in detail 
below in the Description. However, this Summary is not 
intended to identify key or essential features of the claimed 
5 subject matter, nor is it intended to be used as an aid in 
determining the scope of the claimed subject matter. 

DRAWINGS 

10 Various aspects and attendant advantages of one or more 
exemplary embodiments and modifications thereto will 
become more readily appreciated as the same becomes better 
understood by reference to the following detailed description, 
when taken in conjunction with the accompanying drawings, 
15 wherein: 

FIG. 1A is a schematic illustration of an exemplary flow 
imaging system for implementing the concepts disclosed 
herein; 

FIG. IB is a schematic illustration of an exemplary imag- 
20 ing system for implementing the concepts disclosed herein, 
wherein the objected to be imaged are disposed on a plate or 
slide; 

FIG. 1C is a schematic illustration of a readout provided by 
a TDI detector employed in an exemplary flow imaging sys- 
25 tern used in accord with the concepts disclosed herein; 

FIG. 2 is a schematic illustration of an exemplary stereo- 
scopic flow imaging system used in accord with the concepts 
disclosed herein; 

FIG. 3 is a block diagram schematically illustrating the 
30 basic components of an exemplary flow imaging system suit- 
able for implementing the concepts disclosed herein; 

FIG. 4 is a block diagram schematically illustrating a 
method of providing extended depth of field imagery for an 
object; 

35 FIG. 5 schematically illustrates a filter wheel for selec- 
tively positioning an wave front distortion element in a light 
path of a flow imaging system, such that the imaging system 
can be used for both EDF imaging and normal imaging, 
depending on the position of the filter wheel relative to the 
40 light path; 

FIG. 6 illustrates a three-dimensional (3-D) contour of an 
exemplary optical element configured to deform the optical 
wave front of light from an object; 

FIG. 7 graphically illustrates the fractional focal offset as a 
45 function of transmission angle due to a glass parallel plate, 
representing another type of optical element that can be used 
to intentionally deform light from the object; 

FIG. 8 schematically illustrates how the PSF of an imaging 
system affects the quality of an image generated by the imag- 
50 ing system; 

FIG. 9 schematically illustrates the effect that varying 
degrees of defocus have on an image acquired using a stan- 
dard non-extended depth of field imaging method; 

FIG. 10A graphically illustrates an absolute comparison of 
55 through focus modulation transfer functions, at half the 
Nyquist frequency, of a standard non-extended depth of field 
imaging method and a plurality of extended depth of field 
(EDF) imaging methods, as disclosed herein, the EDF meth- 
ods including an exemplary Wave Front Coded (WFC) EDF 
60 method, an exemplary Spherical Aberration EDF method, 
and an exemplary Tilted Object Plane Time Delay Integration 
(TOPTDI) EDF method; 

FIG. 10B graphically illustrates a normalized comparison 
of modulation transfer functions of a standard non-extended 
65 depth of field imaging method (diffraction limited) and the 
WFC EDF method, the Spherical Aberration EDF method, 
and the TOPTDI EDF method; 
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FIG. 11A illustrates the best focus imagery obtained using 
the exemplary imaging system of FIG. 1A, for both standard 
imaging and EDF imaging, before processing the image data 
to correct for errors introduced by the wave front deforma- 
tion; 

FIG. IB illustrates the best focus imagery obtained using 
the exemplary imaging system of FIG. 1A, for various EDF 
techniques, after de-convolution; 

FIG. 12A illustrates the imagery 5 microns away from the 
best focus obtained using the exemplary imaging system of 
FIG. 1A, for both standard imaging and EDF imaging, before 
processing the image data to correct for errors introduced by 
the wave front deformation; 

FIG. 12B illustrates the imagery 5 microns away from the 
best focus obtained using the exemplary imaging system of 
FIG. 1A, for various EDF techniques, after de-convolution; 

FIG. 13 A illustrates a sampling of PSF imagery collected 
using the exemplary imaging system of FIG. 1A, modified to 
implement WFC EDF imaging (modified by introducing a 
phase plate in infinite space); 

FIG. 13B illustrates an exemplary de-convolution kernel; 

FIG. 14 A illustrates the imagery collected using the exem- 
plary imaging system of FIG. 1 A over a 1 6 micron focus pan 
for standard imaging (i.e., not EDF imaging); 

FIG. 14B illustrates the imagery collected using the exem- 
plary imaging system of FIG. 1A modified for WFC EDF 
over a 1 6 micron focus pan; 

FIG. 15 A illustrates peak pixel intensity versus object 
number for image data collected by the exemplary imaging 
system of FIG. 1 A operating in a standard mode (i.e., not EDF 
imaging)) during a step-wise focus pan in which approxi- 
mately 200 objects were imaged at each of nine focus posi- 
tions; 

FIG. 15B illustrates area versus object number for image 
data collected by the exemplary imaging system of FIG. 1A 
operating in a standard mode (i.e., not EDF imaging) during 
a step-wise focus pan in which approximately 200 objects 
were imaged at each of nine focus positions; 

FIG. 16A illustrates peak pixel intensity versus object 
number for image data collected by the exemplary imaging 
system of FIG. 1A operating in an EDF mode) during a 
step-wise focus pan in which approximately 200 objects were 
imaged at each of nine focus positions; 

FIG. 16B illustrates area versus object number for image 
data collected by the exemplary imaging system of FIG. 1A 
operating in an EDF mode) during a step-wise focus pan in 
which approximately 200 objects were imaged at each of nine 
focus positions; 

FIG. 17A illustrates FISH imagery of cells with disomy for 
chromosome Y collected by the exemplary imaging system of 
FIG. 1A operating in a standard mode (i.e., not EDF imag- 
ing); 

FIG. 17B illustrates FISH imagery of cells with disomy for 
chromosome Y collected by the exemplary imaging system of 
FIG. 1A operating in an EDF mode; 

FIG. 18 graphically illustrates how EDF data collected 
using the exemplary imaging system of FIG. 1A operating in 
EDF mode can be used to discriminate single cells from 
debris or cell clusters prior to classifying and enumerating 
chromosomes; 

FIG. 19 A graphically illustrates a gray-scale fluorescence 
image prior to segmentation; 

FIG. 19B graphically illustrates a segmentation mask to 
isolate areas of local maxima after initial segmentation; 

FIG. 19C graphically illustrates a segmentation mask to 
isolate areas of local maxima after morphology segmenta- 
tion; 
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FIG. 20 graphically illustrates an analysis of cellular 
images obtained using the exemplary imaging system of FIG. 
1A operating in a standard mode (i.e., not EDF imaging); 

FIG. 21 graphically illustrates an analysis of cellular 
5 images obtained using the exemplary imaging system of FIG. 
1A operating in an EDF mode; 

FIGS. 22A-22D illustrate randomly selected cell imagery 
obtained using the exemplary imaging system of FIG. 1A 
operating in a standard mode (i.e., not EDF imaging) and 
to corresponding to “Monosomy Refinement” gates (FIG. 22A), 
Monosomy false positive events (FIG. 22B), “Disomy 
Refinement” gates (FIG. 22C), and “Disomy false positive 
events (FIG. 22D); 

FIGS. 23A-23D illustrate randomly selected cell imagery 
15 obtained using the exemplary imaging system of FIG. 1A 
operating in an EDF mode and corresponding to “Monosomy 
Refinement” gates (FIG. 23A), “Monosomy false positive 
events” (FIG. 23B), “Disomy Refinement” gates (FIG. 23C), 
and “Disomy false positive events” (FIG. 23D); and 
20 FIG. 24 schematically illustrates an exemplary computing 
system suitable for use in implementing the method of FIG. 4. 

DESCRIPTION 

25 Figures and Disclosed Embodiments are not Limiting 

Exemplary embodiments are illustrated in referenced Fig- 
ures of the drawings. It is intended that the embodiments and 
Figures disclosed herein are to be considered illustrative 
rather than restrictive. No limitation on the scope of the tech- 
30 nology and of the claims that follow is to be imputed to the 
examples shown in the drawings and discussed herein. 

The concepts disclosed herein encompass a method of 
adding extended depth of field capability to a flow imaging 
system configured for high-resolution imagery (exemplary, 
35 but not limiting parameters include 0.75 NA and 0.5 micron 
pixel size). It thus should be recognized that the term “stan- 
dard image” or “standard imaging” refers to use of an exem- 
plary flow imaging system (described in detail below) that has 
not been modified for EDF imaging. Such a flow imaging 
40 system can combine the speed, sample handling, and cell 
sorting capabilities of flow cytometry with the imagery, sen- 
sitivity, and resolution of multimode imagery with an 
extended depth of field in order to provide a comprehensive 
feature set to more effectively eliminate artifacts and allow 
45 for the complex analysis of the location, distribution, and 
translocation of biomarkers. Standard, non-confocal methods 
of image collection are hindered by extended depth of field 
limitations. The EDF capability described herein is a result of 
modifying an exemplary flow imaging system with an ele- 
50 ment in aperture space to alter the wave front in a determin- 
istic way. The combination of a modified wave front and 
post-processing of the imagery helps to mitigate the spatial 
resolution loss and blurring associated with defocus. The 
result is a 2-D projection of the 3-D cell for each of six 
55 multimode images (it being understood that the use of six 
images are exemplary, and not limiting on the technique) 
acquired at rates 1 00 to 1 000 times faster than confocal image 
stacking techniques. With the extended depth of field 
enhancement disclosed herein, micron-level spatial resolu- 
60 tion can be maintained over the entire cell so that cellular 
structures and probes lying outside the plane of best focus can 
be analyzed with greater accuracy, as demonstrated with 
empirical FISH probe image data discussed in detail below. 

More specifically, there are at least four applications in 
65 which such EDF imagery from flow imaging systems can be 
beneficially employed, including: (1) Cell Activation, such as 
transcription factor NF-eB nuclear translocation; (2) Mecha- 
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nisms of Monoclonal Antibody Drug Action, co-localization 
and compartmentalization; (3) Apoptosis Analysis: differen- 
tial rates of apoptosis in heterogeneous cell samples; and, (4) 
Morphologic cell classification, the identification of cells in 
blood and bone marrow. 

Before discussing the steps employed in one exemplary 
embodiment for implementing the present novel approach, it 
will be beneficial to review an exemplary flow imaging sys- 
tem 150 that can be used to execute this method. FIG. 1A 
illustrates the key components of an optical system employed 
to project light from objects in flow onto a detector that 
employs an exemplary readout for any type of small object 
(although imaging cells represent an exemplary application). 
Objects are hydrodynamically focused into a single-file line 
in a fluidic system (not separately shown), forming a tall but 
narrow field of view. This method enables the lateral dimen- 
sion of the detector to be used for signal decomposition. FIG. 
3 is a block diagram showing the major components of exem- 
plary flow imaging system 150. A key component is a distor- 
tion element 152, which is used to distort the wave front of the 
light collected from an object, in a predictable and largely 
reversible fashion, while expanding the field of view. Refer- 
ring to FIG. 3, the major components of a multi -mode EDF 
flow imaging system include a flow cell or cuvette 151 into 
which object (such as cells) to be imaged are directed, an 
optical distortion element 152 to introduce a well-character- 
ized distortion of light received from the object (such that the 
PSF of the imaging system is substantially invariant over a 
range of focal planes), collection, dispersing, and imaging 
optical elements 153 (to enable the multi-mode imaging such 
as shown in FIG. 1A to be achieved), a detector 154 for 
generating raw image data (preferably, a TDI detector, 
although other imaging detectors can alternatively be 
employed, i.e., non-TDI imaging detectors can also be used), 
and a processor 155 for processing the image data, to enhance 
the image data and at least partially correct for the distortions 
introduced by the distortion element. Such processing can 
comprise a de-convolution that reduces spatial broadening 
and enhances contrast. It should be recognized that such 
processing can be implemented using hardware (e.g., a cus- 
tom processing circuit or an application specific integrated 
circuit (ASIC)), or a combination of hardware and software 
(e.g., a software-driven processor, such as is typically used in 
a computing device, a personal computer being one well- 
known example thereof). It should further be recognized that 
the distortion element may be physically located outside of 
the aperture of the imaging system (i.e. between the flow cell 
and the imaging objective in the case of the spherical aberra- 
tion technique), in aperture space after the imaging objective 
(in the case of the WFC technique), or may be effected within 
the imaging objective itself by adjustment of an aberration 
correction collar. 

Referring now to FIG. 1A, object(s) 99 are hydrodynami- 
cally focused in a flow of fluid directed into a flow cuvette 116 
and illuminated from one or more sides using light sources 98 
and 100. Light is collected from the objects with a high NA 
objective 102, and the light that is collected is directed along 
a light path including lenses 103 A and 103B, and a slit 105. A 
fraction of this collected light is transmitted to an auto-focus 
subsystem 104 and to a velocity detection system 106. It 
should be noted that in connection with a velocity detection 
system 106 that uses a TDI, it is important to ensure the data 
signal produced by the detection system, which is integrated 
over time to increase the signal -to-noise ratio, is properly 
synchronized with the flow of objects through the imaging 
system. In the context of an exemplary implementation, the 
objects are fluorescently labeled beads or fluorescently 
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labeled cells. The extended depth of field capability afforded 
by the present exemplary technique disclosed herein is par- 
ticularly useful in automated chromosome enumeration via 
FISH probing of Jurkat cells, although such use is intended to 
5 be exemplary, rather than limiting on the application of this 
technique. 

Either an optical distortion element 5 A is disposed 
between the objects being imaged and the collection lens, or 
an optical distortion element 5B is disposed in infinite space 
to (that is, at the objective aperture or at a conjugate image of the 
aperture at a subsequent location in the optical system, but 
before the detector). Alternatively, optical distortion may be 
introduced via adjustment of a correction collar on an adjust- 
able implementation of objective lens 102. Only one means of 
1 5 introducing optical distortion is required. The function of the 
optical distortion is to change the light from the object to 
achieve a PSF that is substantially invariant across an EDF, 
such that negative effects of the distortion produced by the 
element can subsequently be removed by signal processing, 
20 to yield an EDF image. 

Yet another technique that can be used to introduce optical 
distortion into light from the object is to use a cuvette/flow 
cell having different optical thicknesses at different locations, 
such that imaging through the different locations of the 
25 cuvette induces different degrees of wave front deformation. 
For example, different faces of the cuvette can induce differ- 
ent levels of distortion, with one or more faces introducing no 
intentional distortion/deformation, with other faces config- 
ured to intentionally deform the optical wave front of light 
30 from the object. Moving the cuvette relative to the imaging 
optical enables the deformation to be selectively induced. An 
optional cuvette manipulator 9 for manipulating the position 
of the cuvette relative to the optical system is shown in FIG. 
1A. Where different faces of the cuvette induce different 
35 levels of deformation, such means will generally rotate the 
cuvette. It should also be recognized that a single face of a 
cuvette can induce different levels of deformation at different 
locations, such that translating the cuvette linearly can induce 
different levels of deformation. In such an embodiment, 
40 manipulator 9 will be configured to translate the cuvette lin- 
early. Those of ordinary skill in the art will recognize that 
many different structural configurations can be used to imple- 
ment manipulator 9, such as stepper motors, linear actuators, 
hydraulics, powered hinges, powered linkages, and others. 
45 The specific configuration is not critical, so long as manipu- 
lation of the cuvette does not introduce additional optical 
errors beyond the intentional deformation, thus the specified 
structures for manipulator 9 should be considered exemplary, 
rather than limiting. 

50 The majority of the light is passed to a spectral decompo- 
sition element 108, which employs a fan-configuration of 
dichroic mirrors 110 to direct different spectral bands later- 
ally onto different regions of a TDI detector 114. Thus, the 
imaging system is able to decompose the image of a single 
55 object 118 into multiple sub-images 120 across detector 114, 
each sub-image corresponding to a different spectral compo- 
nent. In this view, detector 114 has been enlarged and is 
shown separately to highlight its elements. 

Spectral decomposition greatly facilitates the location, 
60 identification, and quantification of different fluorescence- 
labeled biomolecules within a cell by isolating probe signals 
from each other, and from background auto fluorescence. 
Spectral decomposition also enables simultaneous multi- 
mode imaging (brightfield, darkfield, etc.) using band-limited 
65 light in channels separate from those used for fluorescence 
imaging. FIG. 1A illustrates an exemplary flow-based 
embodiment of flow imaging system 150. However, it should 
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be recognized that such an imaging system can be configured 
to collect images of objects on a plate or slide 7, where the 
plate/slide moves relative to the imaging system, instead of 
the flow -based embodiment, as indicated in FIG. IB. 

It should be recognized that other elements (such as a prism 5 
or a filter stack) could be similarly employed to spectrally 
disperse the light, and the dichroic mirrors simply represent 
an exemplary implementation. Flow imaging system 150 can 
employ a prism (not shown) or a grating oriented to disperse 
light laterally with regard to the axis of flow prior to the final 10 
focusing optics, for spectral analysis of each object’s intrinsic 
fluorescence. In yet another exemplary embodiment of a suit- 
able flow imaging system that is contemplated (but not 
shown), a cylindrical final focusing lens can be employed to 
image a Fourier plane on the detector in the cross-flow axis, 15 
enabling analysis of the light scatter angle. These techniques 
for multi-spectral imaging, flow spectroscopy, and Fourier 
plane scatter angle analysis can be employed simultaneously 
by splitting the collected light into separate collection paths, 
with appropriate optics in each light path. For enhanced mor- 20 
phology or to analyze forward scatter light, a second imaging 
objective and collection train can be used to image the par- 
ticles through an orthogonal facet of the flow cuvette 116, 
thereby viewing the objects in stereoscopic perspective with 
no loss of speed or sensitivity. 25 

To analyze the collected imagery, a software based image 
analysis program can be employed. One example of suitable 
image analysis software is the IDEAS™ package (available 
from Amnis Corporation, Seattle, Wash.). The IDEAS™ soft- 
ware package evaluates over 200 quantitative features for 30 
every cell, including multiple morphologic and fluorescence 
intensity measurements, which can be used to define and 
characterize cell populations. The IDEAS™ software pack- 
age enables the user to define biologically relevant cell sub- 
populations, and analyze subpopulations using standard 35 
cytometry analyses, such as gating and backgating. It should 
be understood, however, that other image analysis methods or 
software packages can be employed to apply the concepts 
disclosed herein, and the IDEAS™ image analysis software 
package is intended to be merely one example of a suitable 40 
software for this purpose, rather than limiting on the concepts 
disclosed herein. 

Turning now to FIG. 1C, detector 114 of the exemplary 
flow imaging system shown in FIG. 1 A is implemented using 
a TDI that performs high throughput imaging with high sen- 45 
sitivity. As shown in an exemplary readout 138, the image on 
the TDI detector is read out one row of pixels at a time from 
the bottom of the detector. After each row is read out, the 
signals in the remaining detector pixels are shifted down by 
one row. The readout/shift process repeats continuously, 50 
causing latent image 142 to translate down the detector during 
readout (note the movement of latent image 142 through 
frames T1-T6). If the readout rate of the TDI detector is 
matched to the velocity of the object being imaged, the image 
does not blur as it moves down the TDI detector. In effect, the 55 
TDI detector electronically “pans” the rate at which rows are 
read out to track the motion of an object being imaged. To 
provide optimum results for this technique, it is important to 
accurately measure the velocity of the objects being imaged 
and to employ that measurement in feedback control of the 60 
TDI readout rate. Thus, accurate velocity detection for 
objects moving in flow enables the TDI imaging to be imple- 
mented properly. 

One primary advantage of TDI detection over other meth- 
ods is the greatly increased image integration period it pro- 65 
vides. An exemplary flow imaging system used in connection 
with the present invention includes a TDI detector that has 
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512 rows of pixels, provides a commensurate 512x increase 
in signal integration time. This increase enables the detection 
of even faint fluorescent probes within cell images and intrin- 
sic auto fluorescence of cells acquired at a high-throughput. 

Furthermore, the use of a TDI detector increases measured 
signal intensities up to a thousand fold, representing over a 30 
fold improvement in the signal-to-noise ratio compared to 
other methods disclosed in the prior art. This increased signal 
intensity enables individual particles to be optically 
addressed, providing high-resolution measurement of either 
scattered spectral intensity of white light or scattered angular 
analysis of monochromatic light of selected wavelengths. 

Exemplary flow imaging system 150 can be configured for 
multi-spectral imaging and can operate with, for example, six 
spectral channels: DAPI fluorescence (400-460 nm), Dark- 
field (460-500 nm), FITC fluorescence (500-560 nm), PE 
fluorescence (560-595 nm), Brightfield (595-650 nm), and 
Deep Red (650-700 nm). The TDI detector can provide 1 0 bit 
digital resolution per pixel . The NA of the exemplary imaging 
system is typically about 0.75, with a pixel size of approxi- 
mately 0.5 microns. However, those skilled in the art will 
recognize that this flow imaging system is neither limited to 
six spectral channels nor limited to either the stated NA, or 
pixel size and resolution. 

While the elimination of focus variation in a 2-D projection 
of a cell will likely be beneficial in many applications, it may 
be limiting in others, such as co-localization assays. This 
possibility was a key consideration in the utilization of a 
phase plate for the WFC EDF methodology, because the WFC 
EDF method can be implemented to provide different levels 
of distortion, or disabled completely, by removing or substi- 
tuting optical elements in the system’s aperture plane. 

Another exemplary flow imaging system embodiment is a 
stereoscopic arrangement, as illustrated in a flow imaging 
system 70 of FIG. 2, wherein fluid flow 22 entrains object 99 
(such as a cell, but alternatively, a small particle) and carries 
the object through flow imaging system 70. Light 30 from 
object 99 passes through collection lens 32 that collects the 
light, producing collected light 34, which is approximately 
focused at infinity, i.e., the rays of collected light from col- 
lection lens 32 originating at the same location in the object 
are generally parallel. Collected light 34 enters a prism 36, 
which disperses the light, producing dispersed light 38. The 
dispersed light then enters imaging lens 40, which focuses 
light 42 onto TDI detectors 114a and 114 b. As noted above, 
either optical distortion element 5A or 5B or adjustable objec- 
tive lens 102 (or a cuvette/flow cell configured to introduce 
optical distortion) is included along the optical path between 
each the object being imaged and the detector. It will typically 
be desirable to use the same type of optical distortion tech- 
nique in each optical leg; however, it should be recognized 
that different distortion techniques can be implemented in 
each leg, so long as each signal from each detector is pro- 
cessed appropriately to correct for the intentional distortion in 
each optical leg. 

The use of two different optical legs enables the obj ect to be 
imaged from two different directions, in order to distinguish 
features that would otherwise overlap when viewed from a 
single direction. While this embodiment can also be 
employed for objects on moving substrates, such as micro- 
scope slides, it is particularly useful for analyzing multi - 
component objects in solution flowing through the system, 
such as cells containing FISH probes. Such probes appear as 
point sources of light anywhere within the cell’s 3-D nucleus. 
In some cases, two or more FISH probes may appear in an 
overlapping relationship along the optical axis of the imaging 
system. In such cases, one of the FISH probes may obscure 
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the others, making it difficult to determine the number of 
probes present in the cell. This factor is important in the 
determination of genetic abnormalities such as trisomy 21, 
otherwise known as Down syndrome. Single-perspective sys- 
tems may address this problem by “panning through” the 
object along the optical axis to acquire multiple image planes 
in the object. While this method may be effective, it requires 
a significant amount of time to collect multiple images and 
cannot be readily applied to a cell in flow. The stereoscopic 
imaging system 70 in FIG. 2 includes two TDI detectors 114a 
and 1146, and their associated optical components, as dis- 
cussed above, in connection with flow imaging system 150. 

By positioning the optical axes of collection lenses 32 for 
the two TDI detectors so that they are disposed at an angle to 
each other, for example, about 90 degrees, it is possible to 
separately resolve the FISH spots imaged from two or more 
FISH probes on at least one of TDI detectors 114a or 1146. If 
two or more FISH probes overlap in regard to the image 
produced on one of the detectors, they will be separately 
resolved in the spectrally dispersed images produced on the 
other TDI detector. Further, the use of two TDI detectors in 
flow imaging system 70 in what might be referred to as a 
“stereo or three-dimensional configuration” provides flex- 
ibility in the configuration of each leg of the system, including 
parameters such as the relative TDI readout rates, axial ori- 
entations, inclinations, focal plane positions, and magnifica- 
tion. Multiple cells or other objects may be imaged onto each 
detector simultaneously in the vertical direction. Since the 
objects move in synchronicity with the signal on the TDI, no 
gate or shutter is required to prevent blurring of the image. A 
pulsed or CW light source (without the need for a trigger 
mechanism to time a pulse coincident with particle arrival in 
the field of view) is employed. If a pulsed light source is used, 
the extended field of view in the axis of motion associated 
with TDI detection enables the cell or object in motion to be 
illuminated by multiple light pulses during its traversal 
through the imaging system. In contrast to a frame-based 
imaging apparatus, a TDI system can produce a single un- 
blurred image of the object that integrates the signal from 
multiple light pulses. When a CW light source is used, the 
signal generated by the object will be collected throughout 
the entire traversal of the object through the field of view, 
rather than only a small segment in time when a shutter is 
open. Therefore, the amount of signal collected and imaged 
on the detector in this exemplary embodiment is substantially 
greater than that of the prior art frame-based imaging sys- 
tems. Consequently, the present approach can operate at very 
high throughput rates with excellent signal -to-noise ratio. 

Application of the optical system shown in FIG. 1A in an 
orthogonal configuration as shown in FIG. 2 provides the 
ability to determine the location of a source of light within the 
cell in three dimensions. A single axis EDF system projects 
light from the cell onto a two dimensional plane without the 
blur associated with defocus. Therefore, in X-Y-Z Cartesian 
space, one can readily determine, for example, the location of 
light coming from the cell in the X and Z axes, assuming the 
optic axis is parallel to theY axis. In the Y axis no positional 
information is available. However, if a second EDF optical 
system is positioned orthogonal to theY axis and parallel to 
the X axis, one can further determine the location of light 
coming from the cell in the Y axis. A point of light coming 
from a cell will be imaged onto two detectors simultaneously. 
Since both detectors collect light from the same Z axis, the Z 
location of the light on the detectors provides a reference with 
which to correlate the position of light in all three axes. To 
unambiguously determine the X, Y and Z location of a point 
of light, one need only locate the light in the Z axis in the 


14 

appropriate channel on both detectors, 114a and 1146, and 
then assess the position of the light in the X and Y axes on the 
corresponding detector. For an application involving multiple 
discrete sources of light within the cell, such a FISH spot 
5 enumeration, the techniques disclosed herein may be used to 
unambiguously count spots from the cell. For molecular co- 
localization analyses, the same methodology can be applied. 
However, in this case each molecular species is tagged with a 
different color marker. Therefore, the same process is applied 
to in two channels of the detector. Since each channel is spatially 
registered with the other, one can compare the location of 
light in each channel to assess co-localization of two different 
molecular species. 

Beyond unambiguously locating the position of discrete 
15 sources of light from the cell, the concepts disclosed herein 
can also be used to reconstruct 3-D models of solid bodies and 
surfaces within the cell. This can be accomplished by dividing 
the volume of the cell into a set of voxels with a dimension is 
each axis equal to the pixel size in each axis on the detectors. 
20 The intensity of each voxel in the volume can be determined 
in stepwise fashion. On a given detector, a single pixel, in the 
X-Z plane for example, represents the sum of voxel intensities 
for all voxels at that X-Z location along theY axis. Therefore, 
to determine the signal in each voxel along theY axis (at that 
25 X-Z location), the total signal from the X-Z pixel would be 
apportioned into each voxel along the Y axis in accordance 
with relative proportion of signal present in each pixel along 
the corresponding row on the Y-Z detector. For example, the 
signal for an arbitrary set of voxels, X3Z5Y1, X 3 Z 5 Y 2 , 
30 X 3 Z 5 Y 3 , X 3 Z 5 Y . . . , X3Z5Y100, could be determined as 
follows. The signal for pixel X 3 Z 5 in the third column and 
fifth row on detector X-Z would contain the sum of the signal 
for all voxels listed above. If this sum were 1000 counts and 
all 1 00 pixels on the fifth row of the Y-Z detector contained the 
35 same value, than the 1000 count signal would be distributed 
evenly among all voxels listed. If for example, only the \0 th 
and 11* pixels contained signal, then all voxel signal levels 
would be set to zero except for voxels X 3 Z 5 Y 10 and X 3 Z 5 Y : l . 
The 1000 count signal would then be distributed into those 
40 voxels accordance with the relative signal levels in pixels 10 
and 11 on the fifth row of detector Y-Z. In this manner all 
voxels throughout the volume of a cell could be assigned 
signal levels to construct a 3-D model of the cell. This model 
could then be viewed from any angle, and sliced along arbi- 
45 trary planes, to better visualize the spatial arrangement of 
cellular components and molecules contained within a cell. 

Also illustrated in FIG. 2 are several exemplary positions 
for light sources, which are useful for different purposes in 
connection with flow imaging system 70. Light sources are 
50 disposed so that light 58 emitted from the source travels 
toward the object in a direction that is generally aligned with 
the optical axis of collection lens 32, and the image formed on 
the TDI detectors thus will not include light absorbed by 
obj ect 99 . Light absorption characteristics of the obj ect can be 
55 determined by illuminating the object using these light 
sources. More specifically, in connection with TDI detector 
114a, light source 62 provides illumination of object 99 from 
a direction so that absorption characteristics of the object can 
be determined from the image produced on the TDI detector. 
60 At the same time, light provided by light source 62 that is 
scattered from object 99 can be used to produce a scatter 
image, and spectrally dispersed images on TDI detector 1146. 
Light source 74 can be employed to produce spectrally dis- 
persed and scattered images on both TDI detectors 114a and 
65 1146. If light sources 62 and 72 are of different wavelengths 
and an appropriate filter is provided to block the wavelength 
from the light source aligned with the optical axis of respec- 
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tive collections lenses 32, these two light sources can be used 
for producing scattered light from the object. For example, 
suppose light source 72 produces light of a wavelength A that 
scatters from object 99 and is directed toward TDI detector 
114a. By including a filter (not shown) that blocks a wave- 
length B produced by light source 62, the light at wavelength 
B will not directly affect the images produced on TDI detector 
114a. Similarly, the light from light source 72 would be 
blocked with an appropriate filter (not shown) so that it does 
not interfere with the imaging of light produced by light 
source 62 that is scattered from object 99 onto TDI detector 
1146. 

Epi light source 66 is also illustrated for use in producing 
images on TDI detector 114a in connection with partial 
reflector 68. Light source 64 can be used to generate reflected 
light to produce images on TDI detector 1 14a, while scattered 
light from this source is directed toward TDI detector 1146. 
These and other possible locations of light sources will be 
apparent to those of ordinary skill in the art, as appropriate for 
providing the incident light on the object needed to achieve 
imaging, depending upon the particular application and infor- 
mation about the object that is desired. Moreover, if the WFC 
EDF method that is described below in detail is applied to 
both legs of flow imaging system 70, an accurate 3-D map of 
the cell can be reconstructed. 

While the system of FIG. 2 can be employed to acquire 
non-EDF images (i.e., it can be used without optical distor- 
tion elements 5A or 5B, or adjustable objective 102 or cuvette 
or cover slip configured to introduce optical distortion), the 
use of such elements and the post-image processing to par- 
tially correct for such distortion enables stereoscopic high 
definition EDF imaging to be acquired from objects in flow, 
thereby enabling a large amount of image data to be acquired 
for a large number of objects much more rapidly than is 
possible using confocal microscopy. 

Extended Depth of Field Imaging 

EDF as used herein refers to the capability of imaging more 
parts of an object in focus than could be imaged using an 
unmodified imaging system (i.e., an imaging system not 
modified to achieve the EDF imaging). Such EDF imaging 
can enable all cellular components within a ten micron or 
greater depth of field to be imaged in focus. EDF cellular 
imaging offers an alternative method to developing a confo- 
cal -like image projection with the entire cell in focus simul- 
taneously. One of the issues raised by single-plane image 
capture of microscopic objects is the effect of focus variations 
on the quality of captured imagery. In particular, if the object 
to be imaged has fine structures, which are intrinsically in 
different focal planes, it is not possible to resolve all of the 
corresponding fine detail in a single planar image. The finer 
the details to be imaged, the more important this problem 
becomes, because the size of the smallest features that can be 
resolved varies inversely with the NA of the optical system, 
while the depth of focus shrinks faster, as the inverse square of 
the NA. Thus, EDF imaging can be accomplished at very high 
speed and eliminates the photo bleaching effects associated 
with repeated acquisitions of the cell imagery at different 
planes of focus. EDF imaging can be accomplished in several 
ways. However, the underlying principal involves the forma- 
tion of a PSF that is invariant over an expected range of focal 
positions. For most cell imaging applications, this range is 
approximately 15 microns. The process of achieving a PSF 
invariant to focal position increases the size and changes the 
character of the PSF when compared to the classical best 
focus point spread. The increased size reduces the ability of 
the optical system to generate contrast and resolve image 
detail. However, through de-convolution, the contrast can be 
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largely restored with the benefit of providing “best-focus- 
like” resolution over a greatly enhanced focal range. The end 
result is a high-resolution image of the cell with all features 
simultaneously in focus. 

5 The concepts disclosed herein encompass at least three 
methods to achieve focus invariant PSFs: (1) a WFC EDF 
method using a phase plate, for example, a WAVE FRONT 
CODED™ element provided by CDM Optics, Inc. of Boul- 
der, Colo.; (2) a Spherical Aberration EDF method; and, (3) a 
to Tilted Object Plane Time Delay Integration (TOPTDI) EDF 
method. Basic principles relating to the TOPTDI EDF 
method are described in U.S. Pat. No. 6,583,865. The present 
discussion briefly covers improvements to the TOPTDI 
method. It should be noted that the WFC EDF technique and 
1 5 the Spherical Aberration EDF technique can be distinguished 
from the TOPTDI EDF technique, in that the TOPTDI EDF 
technique acquires data from different focal planes at differ- 
ent times, and thus requires a TDI detector. The WFC EDF 
technique and the Spherical Aberration EDF technique 
20 acquire EDF data simultaneously, and a TDI detector is not 
required. Use of a TDI detector in implementing the WFC 
EDF technique and the Spherical Aberration EDF technique 
is desirable, because the TDI detector increases the amount of 
light (and therefore data) that can be collected from any 
25 object, thereby improving the signal -to-noise ratio of the 
image; however, each different image acquired by the TDI 
detector includes an EDF before the integrated image is pro- 
vided by the detector, in contrast to the TOPTDI implemen- 
tation. 

30 In summary, all three methods result in a PSF that inte- 
grates light from different focal positions in object space, 
making it relatively insensitive to defocus. This property, in 
turn, enables de-convolution of the PSF to remove the spatial 
broadening and contrast loss inherent in the unprocessed 
35 image, thereby increasing image fidelity and creating an “in- 
focus” projected image of the entire cell. However, only the 
WFC EDF method allows for directed tuning of the optical 
wave front to optimize the PSF for EDF imaging. 

FIG. 4 is a block diagram illustrating the method of pro- 
40 viding multi-mode extended depth of field imagery for an 
object. At a step 160, the method begins with deformation of 
the optical wave front of light from an object as shown in a 
step 162. As discussed below in detail, the optical wave front 
may be deformed by one of a phase plate configured to induce 
45 a phase change to light passing through the phase plate, an 
optical element configured to induce a phase change or dis- 
tortion to light passing through the optical element, an adjust- 
able objective lens, or a cuvette having different thicknesses 
at different locations through which the object can be imaged 
50 (in at least one embodiment, the cuvette/flow cell includes a 
face configured to induce distortion, and a face that does not 
induce distortion, such that rotating the cuvette relative to the 
imaging optics enables the distortion to be selectively 
induced). This deformation is performed in such a way that a 
55 PSF does not vary substantially across an extended depth of 
field. Some cellular objects have fine structures that are in 
different focal planes, and the application of the deformation 
of the wave front enables all image features within the 
extended depth of field across the different focal planes to be 
60 clearly in focus (at least after image processing has reversed 
the distortion effects), because the deformation defocuses and 
expands the depth of field, thereby enabling light from the 
different focal planes to be simultaneously collected. The 
method continues in a step 163, wherein the deformed or 
65 modified light is dispersed into a plurality of different light 
beams. Such spectral dispersion enables a dispersed image 
(or a plurality of spectrally distinct images) of the object to be 
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acquired, generally as indicated in FIG. 1A. An imaging lens 
is used to generate a dispersed image (or a plurality of images) 
of the object in a step 164, and the dispersed image is detected 
in a step 166. The detector of FIG. 1A can thus generate either 
a plurality of different spectral images (one image per chan- 5 
nel), or a single dispersed image (comprising a plurality of 
spectrally dispersed images). 

Optional step 168 indicates that the next step is to deter- 
mine the PSF of the imaging system that produced the 
deformed optical wave front. While this step is required to 10 
process the raw image data to generate the best quality EDF 
image, it should be recognized that such a step may not be 
implemented by a user in some imaging systems, since it 
could be implemented by a manufacturer of the imaging 
system, and stored as a known parameter (thus the arrow from 1 5 
step 168 to start). The PSF of the imaging system including 
the optical distortion element need only be determined once, 
as long as the configuration of the imaging system remains 
unchanged. Once a change is made to the optical configura- 
tion of the imaging system that changes the imaging system’ s 20 
inherent PSF, the PSF for the modified system would need to 
again be determined. In a step 170, the image data can be 
de-convolved using the PSF to reduce negative effects of the 
wave front distortion. Then, the extended depth of field image 
for the object can be produced in a step 172, and the method 25 
is complete as shown in a step 174. 

Simulation of the WFC EDF Method 

The WFC EDF method involves imparting a deformation 
in the optical wave front of the object via the addition of an 
optical or distortion element such as a phase plate (or prefer- 30 
ably the WAVE FRONT CODED™ element provided by 
CDM Optics, Inc. of Boulder, Colo.) in the aperture plane of 
the optical system. The deformation causes light from differ- 
ent focal planes corresponding to a single lateral position in 
object space to be imaged on the detector plane simulta- 35 
neously. A significant advantage of the WFC EDF method 
over the TOPTDI EDF method is the ease with which the 
system can be converted from standard imaging to EDF imag- 
ing. The conversion requires the insertion of the WFC ele- 
ment in the aperture space of the system. The exemplary flow 
imaging system was designed to place an external image of 40 
the aperture in an easily accessible location. For example, a 
six -position software controlled aperture wheel 157 is shown 
in FIG. 5, which can be readily disposed in the exemplary 
imaging system between the objective and the detector. In an 
exemplary embodiment, the position of the filter wheel can be 45 
controlled remotely, so that the internal components of the 
imaging system do not need to be accessed to selectively 
switch the imaging system between a standard imaging mode 
and an EDF imaging mode. Of course, the image processing 
required in standard mode and EDF mode will be different. 50 
In order to simulate the WFC EDF method, a phase plate 
was modeled and consists of an optically clear element hav- 
ing a two-axis cubic waveform, where the axes cross orthogo- 
nally. A phase plate is an optical component of transparent 
material placed in the aperture space of an optical system. The 55 
phase plate is optically clear and has slight variations in its 
thickness in order to retard or advance the phase of the wave 
front relative to the un-deviated wave front. CDM Optics, Inc. 
has developed the ability to construct phase plates with arbi- 
trary surface shapes and micron-level fabrication tolerances. 
These phase plates can be used to induce deformations in the 60 
optical wave front to potentially provide a more consistent 
PSF through an extended focal range. Thus, the slight varia- 
tions in thickness across the plate’s surface serve to retard or 
advance the phase of the wave front. From a geometric per- 
spective, the angular changes in the surface of the element 65 
cause ray paths to deviate from their original course to image 
light from different points along the optic axis for a given 


lateral position in the image plane. For this simulation, an 
element was modeled with a sag of the form shown in Equa- 
tion 1, where n=3. 




A coefficient a w =0.000122 was selected to generate 
approximately five waves of peak to valley wave front error 
over the aperture. The element shown as a 3-D rendering in 
FIG. 6 contains about 6.6 microns of total sag. The element 
was modeled at an exposed aperture stop in the exemplary 
flow imaging system to generate the PSFs used in the subse- 
quent analysis as described below. The results are summa- 
rized in FIGS. 10A and 10B, described in detail below. 
Simulation of the Spherical Aberration EDF Method 

The spherical aberration EDF method involves imparting 
spherical aberration in the wave front by inserting a distortion 
element between the object and the objective lens in order to 
induce spherical aberration into the collected imagery. Useful 
distortion elements include a cover slip (or parallel plate), a 
custom objective with a correction collar, a custom cuvette 
having different optical properties in different parts of the 
cuvette, and switchable optical elements in the image collec- 
tion path. Spherical aberration causes different regions in the 
aperture to focus at different points along the optical axis so 
that points from multiple focal planes in object space are 
imaged onto one detector plane. 

Prior to providing a more detailed discussion of the more 
complete simulations of the optical system performed with 
the parallel plate in place, it may first be helpful to present an 
approximate calculation of the effect of the introduction of a 
parallel plate of glass on the variation of the focal positions of 
light entering a microscope objective. Assume that there is a 
passage of a ray of light R from an object 0 through a parallel 
plate of glass of thickness t. The ray R leaves the object 0 at an 
angle © to the optical axis. As shown in Equation 2, it is bent 
according to Snell’s Law to an angle (|>: 


where n is the refractive index (approximately 1.5) inside the 
glass. 

Upon leaving the glass, it is bent back to its original angle. 
While inside the glass, as shown by Equation 3, it has traveled 
a distance y further from the optical axis than it was at its entry 
point: 


r _./sin0\-i (3) 

y = rtan0 = rtanjsin ^ JJ 

Tracing the exiting ray R back to where it appears to origi- 
nate on the optical axis, Equation 4 shows that the focal 
displacement z' due to the presence of the glass plate is: 


One useful limit to consider is the case when 0 is very 
small. In this case it can be shown that: 


limY = r(l — 1 /re) 

0->O 
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The spherical aberration of the optical system is caused by 
the fact that z' does not remain constant as 0 ranges from zero 
up to the NA of the microscope objective, which represents 
the largest angle ray the objective will accept. The objective 
used in the exemplary flow imaging system has an NA of 0.75 . 

FIG. 7 shows the fractional focal offset z'/t as a function of 
0, assuming a nominal glass refractive index n of 1 .5. Along 
the optical axis, z'/t is about 0.33, while at the maximal angle 
for an NA of 0.75 (the NA of the exemplary flow imaging 
system), z'/t is about 0.49. The difference between these two 
fractional focal displacements is approximately 0.16. To set 
this fractional difference equal to a 10 micron focal depth 
would require a glass thickness of 1010.16=63 microns. In 
this experiment, such a thin slip of glass was not readily 
available, and instead, a commercially available 110 micron 
thick cover slip was used. The focal offsets introduced by this 
cover slip varied over a range of 17 microns, which is larger 
than ideal for covering the depth of a cell nucleus, but still 
well in the useful range for experimentation. 

Now, returning to the simulation, a desirable feature of the 
Spherical Aberration EDF method capability is selectability, 
where the spherical aberration can be introduced or not, 
depending upon the requirements of the particular assay that 
is being run. The EDF algorithm can then be applied only to 
imagery collected with the EDF hardware in place. A select- 
able EDF can be achieved in a number of ways in hardware. 
First, an insertable cover slip can be disposed between the 
objective and the cuvette. In an empirical study, the cover 
glass was held in place on the face of the cuvette by a drop of 
alcohol to form a temporary rigid optical contact. The intro- 
duction of the cover glass creates a known aberration in the 
optical system, known as spherical aberration. In brief, 
spherical aberration causes light rays departing a single 
source at different angles relative to the optic axis to focus at 
different positions along the optic axis. The immediate result 
is that imagery becomes more fuzzy. A more subtle effect is 
that the imagery is less dependent on the exact focal position 
of the lens relative to the source. If the imagery is enhanced 
digitally to reduce the fuzziness via de-convolution, while 
preserving the independence from focal position, an instru- 
ment with enhanced depth of field is achieved, at the cost of 
losing some of the signal-to-noise ratio characteristic of in- 
focus data from the original instrument. Issues include preci- 
sion motion requirements and the need to establish and main- 
tain precise angular alignment of the cover slip surfaces 
orthogonal to the optical axis in the very limited working 
distance (0.5 mm) between the cuvette and objective avail- 
able in the exemplary imaging system. 

In another Spherical Aberration EDF embodiment, a cus- 
tom objective with a motorized spherical aberration correc- 
tion collar can be utilized. Potential problems with such an 
embodiment include the need to provide a custom optical 
design, and development of the objective lens and the 
mechanical interface to drive the correction collar. 

In a further Spherical Aberration EDF embodiment, a cus- 
tom cuvette with different optical thicknesses may be pre- 
sented to the image collection optical path. Issues with such 
an embodiment include tight fabrication tolerances on the 
cuvette wall thicknesses and face perpendicularity, precision 
motion control within the optical alignment requirements, as 
well as maintenance of the interface to the fluidic system in 
the flow cell/cuvette assembly. 

In yet another exemplary Spherical Aberration EDF 
embodiment, switchable optical elements in the image col- 
lection optical path may include the final focus lens to the 
camera/detector, which can be designed with residual spheri- 
cal aberration and disposed in place of the standard focusing 
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lens during EDF imaging. Issues include the optical design 
and fabrication of the spherically aberrated lens to maintain 
parfocality with the standard lens and the motion control 
system for swapping the lenses in the optical path. 

5 Varying degrees of spherical aberration were modeled to 
determine the best trade off between contrast loss and depth 
of field expansion. Evaluation of the PSFs at various focal 
positions provides a qualitative understanding of the limita- 
tions of the Spherical Aberration method. Ideally, the PSF 
to would remain fairly consistent over the focal range of interest. 
To simulate the Spherical Aberration EDF method, the exem- 
plary flow imaging system was modeled with a decreased 
flow cuvette thickness to add 1.8 waves of peak- to -peak 
spherical aberration after refocusing. This optical configura- 
15 tion was then used to model the PSF at the various focus 
locations required for subsequent analysis. The results are 
summarized in FIGS. 1 OA and 1 OB, described in detail below. 
Simulation of the TOPTDI EDF Method 
The TOPTDI EDF method can be employed in concert 
20 with the TDI detection methods used in the exemplary flow 
imaging system described in U.S. Pat. No. 6,583,765, the 
drawings and specification of which are hereby specifically 
incorporated herein by reference. In this method, the object 
plane (or detector plane) is tilted such that during the image 
25 integration process, the cell scans through a continuous range 
of focus positions. In other words, the focal plane is tilted 
relative to the axial flow of the cells such that light from 
multiple focal planes in object space is integrated during 
image collection. Either the detector can be tilted relative to a 
30 non-tilted flow path for objects being imaged, or a cuvette 
with a tilted flow path and means for optically correcting for 
the coma and astigmatism that will be introduced by the tilted 
reflective surface of the cuvette wall and air interface can be 
used. In addition, a custom-designed cuvette that has a tilted 
35 channel can be employed, thereby eliminating the concern 
with respect to the astigmatism and coma by ensuring that the 
cuvette is orthogonal to the collected light. Only the water/ 
glass cuvette is non-orthogonal, providing a decrease in opti- 
cal aberrations. Introduction of an optical component such as 
40 an optical wedge, which effectively tilts the image plane with 
respect to the camera, may also be utilized. Alternatively, a 
convolution filter can be used in the beam path, and the known 
de-convolution algorithm may be utilized to correct for the 
astigmatism and coma effects. 

45 The technique of modeling a tilted detector methodology 
to implement a real time “pan through” of focal planes for 
each object during the TDI integration distance showed 
potential; however, empirical studies indicated it requires 
more than a 45 degree tilt of the detector for less than 7 
50 microns of pan through. In addition to implementation diffi- 
culties, this degree of tilt induced an anamorphic pixel aspect 
ratio and decreased collection efficiency. Further empirical 
studies were performed to investigate tilting the object plane 
less than the three degrees, in order to pan through 1 0 microns 
55 of focal positions. Unfortunately, initial modeling studies 
indicated that three degrees of tilt at the air glass interface of 
the cuvette imparted an unacceptable amount of coma and 
astigmatism to the wave front. 

As an alternative, an improved implementation of the 
60 tilted-plane methodology has been developed to achieve the 
desired EDF performance, without the introduction of exces- 
sive off axis aberrations. This method utilizes optical fabri- 
cation techniques developed for precision prism manufactur- 
ing, to polish the desired tilt angle into the cuvette front 
65 surface, relative to the cuvette flow channel, which enables 
the air/glass interface to remain orthogonal to the objective 
optical axis, while the three degree tilted surface is placed at 
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the glass/water interface, thereby substantially reducing the 
residual coma and astigmatism, since the index or refraction 
mismatch is reduced. This TOPTDI system was modeled, and 
PSFs were determined for the matrix of field heights and 
shifted object positions shown below: 


Object Field 
Height (um) 

Object Shift 
Best Focus +5 um 

Object Shift 
Best Focus Position 

Object Shift 
Best Focus -5 um 

128 

+10 

+5 

0 

85.3 

+8.332 

+3.332 

-1.668 

42.7 

+6.668 

+1.668 

-3.332 

0 

+5 

0 

-5 

-42.7 

+3.332 

-1.668 

-6.668 

-85.3 

+1.668 

-3.332 

-8.332 

-128 

0 

-5 

-10 


Using the individual PSF from each row within a given 
column in the matrix above, a PSF was synthesized for a given 
TOPTDI integration path. These PSFs were then used to 20 
compute the through focus MTF plots for the comparison of 
methodologies. The results are summarized in FIGS. 10A and 
10B, as described in detail below. 

Processing of Raw Image Data to Produce the Extended 
Depth of Field Image 

Within limitations, the blur associated with the wave front 
distortion inherent in EDF imaging can be removed through 
post-processing of the image using de-convolution. De-con- 
volution is the process of enhancing the contrast of a system 30 
over some range of frequencies, usually with the objective of 
overcoming some degradation that has occurred in producing 
the image data. The difficulty with this procedure is that noise, 
which is often present at those frequencies, is amplified along 
with any real signal that may be present. Because of this 35 
difficulty, it is usually not desirable to attempt to achieve a 
perfect de-convolution, which would reconstruct a signal 
exactly as it was before the degradation. Instead, the attempt 
should be made to achieve some reasonable level of recon- 40 
struction that does not result in too large an amplification of 
the noise. 

Before discussing the exemplary de-convolution methods 
utilized in processing an image acquired after deforming the 
optical wave front of light from an object to achieve an EDF, 45 
it may be useful to first discuss the spatial resolution and 
depth of focus in optical systems. Diffraction causes a point 
source of light to spread out when imaged by an optical 
system. The resulting intensity pattern, which is called an 
Airy disk, appears as a bright central spot surrounded by a 50 
series of alternating light and dark rings. The intensity pattern 
is a projection of the PSF of the optical system onto a flat 
plane. A PSF that produces an Airy disk having a smaller 
diameter and most of its energy concentrated within the cen- 
tral spot results in a higher spatial resolution. As obj ects move 55 
from the best plane of focus, the Airy disk diameter increases, 
and the energy spreads out into secondary and tertiary rings, 
covering a larger area, resulting in relatively poorer spatial 
resolution. At best focus, the radius (5) of the central bright 
spot of the Airy di sk i s a function the numerical aperture (NA) 60 
of the optical system and the wavelength (X) of light compris- 
ing the image, as defined by the following equation: 


_ 0.62A (6) 65 

" NA 
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The classical depth of field A of an optical system varies 
inversely as the square of the numerical aperture as defined by 
the following equation: 

A 0.5A (7) 

NA 2 

For a typical moderately high-resolution objective (0.75 
NA) used in the center of the visible spectrum (550 nm), the 
diffraction limited resolution and the depth of focus as 
defined by Equations 6 and 7 are 0.45 microns and +/-0.49 
microns, respectively. As illustrated in FIG. 8 , the process of 
imaging is the mathematical equivalent of convolution. The 
spatial and intensity information contained within the object 
(250 line pairs/mm bar target) as shown in the leftmost por- 
tion of FIG. 8 is convolved with the PSF of the optical system, 
resulting in the image shown in the rightmost portion of FIG. 
8 . The image appears very similar to the object, but some 
contrast is lost and the edges of the bars are not as sharp as in 
the original object. This result is caused by the signal content 
in the original object being spread out due to the PSF of the 
optical system. The lower thumbnail images in FIG. 9 dem- 
onstrate the effect of defocus on both the PSF and the result- 
ing imagery as the focus changes. At 1 p (micron) of defocus, 
blurring becomes evident and by 4 g of defocus, the optical 
system has lost the ability to resolve the individual bars in the 
target. By 8 p of defocus, the bar target is unrecognizable and 
suffers significantly diminished intensity. Accordingly, when 
imaged by a 0.75 NA optical system, a cellular feature such as 
a FISH spot having an area of less than one micron and 
located six microns away from the plane of best focus will 
blur into an area covering more than 100 microns, rendering 
it unrecognizable to a human observer and making automated 
detection and enumeration difficult at best. This result can 
occur when viewing FISH probes located at the periphery of 
the nucleus. 

Confocal image stacking techniques avoid this problem by 
synthesizing an image of the cell with all features simulta- 
neously in focus via the collection of multiple images of the 
cell at different focal planes. At each focal position, an image 
is collected by scanning a spot of illumination over the object 
with a conjugate pinhole located at an intermediate image 
plane in the collection system. The conjugate pinhole sub- 
stantially eliminates light from objects outside the focal 
plane, providing a crisp image of the object structures in the 
immediate focal plane. By applying image reconstruction 
algorithms to the stack of imagery, a high-resolution compos- 
ite image can be generated with the entire cell in focus on a 
single plane. 

As discussed above, the convolution process inherent in 
imaging can be “undone” through post-processing of the 
image using de-convolution. This effect can be visually illus- 
trated by reversing the process shown in FIG. 8 , where the 
PSF can be “removed” via de-convolution from the image in 
the rightmost portion of FIG. 8 , such that the image appears 
more like that of the actual object (the leftmost portion of FIG. 
8 ). With good foreknowledge of the PSF, de-convolution 
algorithms can be applied to an image to minimize the effect 
of optical system performance limitations, resulting in a bet- 
ter representation of the original spatial and intensity content 
of the object. This process works well where there is a high 
signal-to -noise ratio in the image, and the object is a two- 
dimensional planar structure with very little depth along the 
optic axis, such as a semiconductor photo-mask, a printed 
page, or the bar target shown in FIG. 8 . However, in cell 
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analysis applications, the objects being analyzed are inher- 
ently 3-D with respect to the depth of field of the optical 
system. The resulting image of a cell on a detector is com- 
posed of many different degrees of point spread depending 
upon the location of a particular cell structure or probe, with 
respect to the plane of best focus. The presence of multiple 
PSFs within the image substantially impairs the de-convolu- 
tion process. Notwithstanding, it should be noted that 3-D 
de-convolution of multiple PSFs has been successfully 
applied to image stacks from standard fluorescence micro- 
scopes; however, the process still requires the collection of 
multiple images of the same cell taken at various positions 
along the optical axis. 

Evaluation of EDF Methods & De-Convolution Using Modu- 
lation Transfer Functions 

A convenient method to theoretically evaluate the expected 
performance of the various EDF methods described above 
(WFC EDF, Spherical Aberration EDF, and TOPTDI EDF) is 
to compare their modulation transfer functions (MTF). The 
typical MTF plot provides a quantitative assessment of con- 
trast over a range of spatial frequencies. For the comparison 
of the EDF methods discussed above, a single spatial fre- 
quency was chosen, and curves were generated for different 
focus positions. A through-focus MTF plot shows the behav- 
ior of the decreasing contrast function on either side of the 
best focus position. The exemplary flow imaging system uti- 
lizes a pixel size of 1 8 microns at the detector, corresponding 
to a maximum sampled spatial frequency of 27.8 line pairs/ 
mm at the detector plane. The through- focus MTF plots were 
calculated at approximately half the maximum resolvable 
spatial frequency, or 14 line pairs/mm (500 line pairs/mm in 
object space), over a focal range of +/-10 microns in object 
space. The optimal performance for an ideal system would be 
a flat response (i.e., a constant MTF) with maximum contrast 
over the widest focal depth. FIGS. 10A and 10B show a 
family of curves representing contrast versus focus for the 
various EDF methods (without PSF de-convolution), as well 
as the non-EDF version of the exemplary imaging system. 

As shown in FIG. 10 A, the standard non-EDF system 
(solid line) provides the best contrast at the plane of focus. 
However, the contrast falls off rapidly as the focal position 
changes. At 2.5 microns of defocus, the standard system 
provides no contrast in the image. Numerous null contrast 
points are observed throughout the plotted focal range. The 
TOPTDI (continuous long dash) EDF method integrates light 
from focal planes over a range of -5 to +5 microns from best 
focus. At best focus, the contrast is about 0.2, which is less 
than one-third of the standard in-focus contrast, but the 
TOPTDI EDF contrast remains relatively constant over a 
much wider range of focal positions. The WFC EDF method 
provides slightly lower contrast than the TOPTDI EDF 
method, but with a greater enhancement to the depth of focus. 
The Spherical Aberration EDF method sacrifices more con- 
trast than either of the other EDF methods modeled here, 
while providing less improvement to the depth of field. It also 
exhibits a classical non-symmetrical behavior about the best 
focus position. The lower plot in FIG. 10B illustrates a 
“modulation normalized” view of the same data, which more 
clearly shows the relative depth of field enhancements pro- 
vided by each method. 

FIGS. 11 A and 12 A show simulated point source imagery 
generated for the various EDF methods using collection 
parameters associated with the exemplary flow imaging sys- 
tem, including an 18 micron pixel size (0.5 microns in object 
space), 0.75 counts of random noise per pixel, and the corre- 
sponding PSFs for the various focus positions. FIG. 11B 
shows the results of ten iterations of a Richardson-Lucy de- 
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convolution algorithm applied to each of the EDF images, 
using the best focus PSF as the de-convolution kernel. The 
peak intensity (based on a 10 bit A/D conversion) for each 
thumbnail image is listed above each image. As demonstrated 
5 in the upper thumbnail images in FIG. 11B, the PSF de- 
convolution process can recover a high degree of image con- 
trast (673 counts for TOPTDI at 5 microns of defocus versus 
801 counts at best focus in the non-EDF image). The display 
of each image is scaled for visualization purposes, so that the 
to brightest pixel in the image appears black, and the lowest 
intensity in the background appears white, causing the back- 
ground noise to appear higher in the cases where the PSF 
imagery has a lower peak intensity, particularly in the case of 
standard imagery at five microns of defocus. It should be 
15 understood in all cases the noise level in all pre-de-convolved 
imagery is the same. 

The simulated imagery illustrates the effectiveness of both 
the TOPTDI EDF and WFC EDF methods in maintaining a 
constant PSF over an extended focal range. The results are 
20 particularly striking when comparing the de-convolved EDF 
imagery to the standard imagery at five microns of defocus . In 
FIG. 12 A, the peak intensity in the standard image drops to 1 9 
counts at the defocus position, while both the TOPTDI EDF 
and WFC EDF methods produce peak intensities in excess of 
25 350 counts (FIG. 12B), resulting in an increase in contrast of 
more than 30 fold (340 counts vs. 9 counts) for both of these 
EDF methods (over standard imaging). The imagery in the 
WFC EDF method exhibits some asymmetrical horizontal 
and vertical artifacts after processing. However, the artifacts 
30 are attenuated by more than an order of magnitude in com- 
parison to the primary image. Optimization of this first-gen- 
eration WFC element and de-convolution kernel is expected 
to further reduce these artifacts. The induced spherical aber- 
ration method fares better under defocus conditions than the 
35 standard optical system, but exhibits much lower contrast 
with defocus than the other two EDF methods: 151 counts 
(FIG. 12B) vs. -350 counts (FIG. 12B). 

Empirical Evaluation of the WFC EDF Method Using Engi- 
neered Bead Samples 

40 Simulation of the various EDF methodologies described 
above offered insights into the advantages and disadvantages 
of each of the alternative EDF methods. The WFC EDF 
method was chosen for implementation in an EDF modified 
version of the empirical imaging system of FIG. 1 A, based on 
45 its high level of performance in modeling, its flexibility in 
implementation, and because it offered the ability to custom 
tailor the degree of depth of field extension to specific appli- 
cations by employing a choice of any one of multiple WFC 
elements, each having a different sag function. 

50 To implement and empirically test the WFC method, a 
crossed cubic WFC element was procured (available from 
CDM Optics, Inc., Boulder, Colo.) The element was installed 
at an external aperture stop in the exemplary flow imaging 
system, and the PSF of the modified imaging system was 
55 measured by running a sample of 200 nanometer diameter 
fluorescent beads (Invitrogen, Fluo Spheres™ carboxy late- 
modified micro spheres, 0.2 pm, yellow-green, 505/515 2% 
solids, F-8811), which was prepared at a 10,000:1 dilution, 
and run on the modified flow imaging system. Such beads are 
60 sufficiently small relative to the pixel size and diffraction 
limited spot size of the exemplary imaging systems optics so 
as to be considered as point sources. FIGS. 13 A and 13B 
show a small sampling of the PSF imagery collected during 
the run. The form of the PSF corresponds very closely to the 
65 modeled imagery shown in FIG. 11B. Approximately 1,000 
bead images were collected and processed to generate the 
composite PSF shown in FIG. 13B. Processing included a 
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linear interpolation of each image to remove shift variance 
caused by under- sampling, and then spatially aligning each 
image such that the highest intensity appeared in the same 
pixel for each image. A mean intensity was then calculated for 
each pixel to generate the composite PSF used in the subse- 
quent image de-convolution step. 

To further evaluate the exemplary flow imaging system 
with WFC EDF imaging, a simple test was devised using 2.5 
micron diameter fluorescent beads (as manufactured by Invit- 
rogen Inc., Linear Flow Green), 0.1% intensity. A file was 
collected containing imagery from nine focus positions 
spaced 2.0 microns apart. The exemplary flow imaging sys- 
tem’s auto-focus control was first enabled to establish the 
nominal best focus for the beads. The auto-focus control was 
then disabled, and the stage was positioned at -8.0 microns 
from best focus. Approximately 200 objects were imaged in 
darkfield and fluorescence modes at each of the nine focus 
positions (-8, -6, -4, -2, 0, 2, 4, 6, and 8 microns from best 
focus), resulting in a file containing 3,600 images over a pan 
range of 1 6 microns. Two test files were collected, one using 
standard imaging, and one using WFC EDF imaging. 

FIGS. 14 A and 14B show an image gallery of 8 pairs of 
consecutively selected beads (1 6 beads total) from each inter- 
val of the focus pan. Darkfield (blue) and fluorescence images 
(green) are shown for each bead. A gallery of bead images 
collected using standard imagery are shown in FIG. 14A, and 
a gallery of bead images collected using WFC EDF imaging 
are shown in FIG. 14B. As is clearly apparent in FIG. 14B, the 
WFC EDF imagery maintains a much higher degree of focus 
over the pan range. Focus blur is present in the standard 
imagery (FIG. 14 A) at +1-2 microns of defocus in both dark- 
field and fluorescent images (objects 701, 702, 1102, and 
1103). At +/-4 microns of defocus, blurring is significant 
(objects 500, 501, 1306, and 1307), and the bead imagery 
exhibits a marked decrease in peak intensity, as well as a large 
change in apparent area. By +/-8 microns of defocus, bead 
images (objects 100, 101, 1707, and 1708) become difficultto 
discriminate from the background. 

In marked contrast, the WFC EDF imagery of FIG. 14B 
maintains consistent image characteristics throughout the 
focus pan with both the bead area and intensity remaining 
relatively constant throughout the pan. This result is particu- 
larly evident in the green fluorescent bead imagery shown in 
the right hand column of FIG. 14B (in a full color image 
channel 3 of FIG. 14B corresponds to green spectral images). 
There are some artifacts present at higher levels of defocus in 
the form of horizontal and vertical lines emanating from the 
primary bead image and directed toward the top and right 
hand side of the page. The artifacts largely resemble those 
generated in the simulation and exhibit much lower intensity 
than the primary image. These artifacts are a result of slight 
changes to the PSF with focus and can be minimized with 
optimizations of the WFC element and de-convolution ker- 
nel. Modeling of the non-orthogonal nature of the artifacts 
has shown that they are also due in part to residual un-cor- 
rected spherical aberration in the optical system. The dark- 
field imagery appears similar in nature to the fluorescence 
imagery; however, it exhibits stronger de-convolution arti- 
facts, especially at high levels of defocus. This result may be 
due in part to the fact that the de-convolution kernel was 
generated from the fluorescent imagery. Future optimizations 
will include channel specific kernels, balancing of in focus 
and out of focus imagery for kernel generation, elimination of 
residual spherical aberration and optimized WFC waveforms; 
each of which will reduce artifacts. 

FIGS. 15A, 15B, 16A, and 16B provide a quantitative 
analysis of the entire image set from which the imagery in 
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FIGS. 14A and 14B was selected. In these Figures, the peak 
pixel intensity and area of each object are plotted against 
object number for both standard images (FIGS. 15 A and 
15B), and EDF images (FIGS. 16A and 16B), where each dot 
5 in a dot plot represents a single bead. A total of 1 ,800 obj ects 
were imaged, including approximately 200 objects acquired 
at each of nine focal positions, with each focal position sepa- 
rated by 2 microns. Object #1 and object #1800 are therefore 
spaced 16 microns apart, with the best focus position corre- 
10 sponding to object numbers in the range of 800-1,000. The 
best focus position was gated using the regions “In Focus pk” 
or “Infocus Area,” with the accompanying mean values for 
the gated data shown below the corresponding dot plot. In a 
similar manner, beads from the +/-8 micron focus positions 
15 were also gated. 

Referring to the dot plots of FIGS. 15 A and 16A (i.e., the 
upper portion of each respective Figure), the dot plot for the 
standard image set (the upper portion of FIG. 15 A) exhibited 
nearly a 14-fold decrease in average peak intensity (641 
20 counts vs. 44 counts) between the best focus and the “8 pm 
defocus” positions. In contrast the EDF dot plot (the upper 
portion of FIG. 16 A) showed only approximately a 2-fold 
decrease in average peak intensity (613 counts vs. 287 counts) 
over the same focus pan. Allowing for the increased focal 
25 range and the larger bead size, these results were consistent 
with the theoretical models for the behavior of peak intensity. 

Referring to the dot plots of FIGS. 15B, and 16B (i.e., the 
upper portion of each respective Figure), the fluorescent EDF 
imagery of FIG. 16B exhibited a consistent area of approxi- 
30 mately 24 pixels throughout most of the focus range, rising to 
32 pixels at the +8 micron defocus position. In contrast, the 
dot plot of the standard imagery (the upper portion of FIG. 
15B) exhibited an increase in area of almost 1 4 times, from 32 
pixels to 437 pixels at -8 microns of defocus. Using the area 
35 and peak intensity as figures of merit, the exemplary flow 
imaging system with WFC EDF imaging demonstrates 7-14 
times better feature consistency through a depth of field cov- 
ering the majority of most prokaryotic and eukaryotic cell 
diameters. 

40 A statistical analysis of the noise contained in the imagery 
was performed by evaluating the standard deviation in back- 
ground signal outside the bead image for each individual 
object. An analysis of over 2,100 objects for each focus pan 
indicates the median standard deviation in background signal 
45 is 0.97 and 1.30 counts, respectively, for the standard and 
EDF focus pans (identified by a in each of FIGS. 15 A and 
16A). The increase in noise of 0.33 counts will degrade the 
signal-to-noise ratio and therefore, negatively impact the sen- 
sitivity of the instrument. However, in comparison to standard 
50 imaging, the degradation will be greatest for objects located 
at the best plane of focus. For objects located away from the 
best plane of focus, the increased signal maintained via EDF 
should more than offset the increase in noise. Sensitivity 
studies of the standard exemplary flow imaging system dem- 
55 onstrate sensitivity superior to standard flow cytometry and 
indicate an ability to detect as little as 50 molecules of equiva- 
lent soluble fluorescein. Further empirical studies of EDF 
imaging will include a detailed sensitivity study of the EDF 
collection mode. 

60 Exemplary Post-Processing De-Convolution of Imagery 

In this empirical study, imagery captured in the WFC EDF 
mode was post processed using a Richardson-Lucy (R-L) 
iterative de-convolution algorithm to restore fidelity. Starting 
with a good measurement of the PSF (as modified by the 
65 WFC element), the technique seeks to maximize the likeli- 
hood of the de-convolved image by using the Expectation 
Maximization (EM) algorithm. Specifically, it assumes an 



US 8,005,314 B2 


27 

undistorted image f which is convolved with a PSF h where n 
denotes the noise associated with the image. Then, EDF 
modified image g is given by the following equation: 

g=h@f+n ( 8 ) 5 

where (x) is the convolution operator. The R-L algorithm 
attempts to reconstruct f using the following relationship: 

(9) 

l h®fk ) 

and where f k is the estimate of f after k iterations, and * is the 
correlation operator. Stability is maintained and convergence 
achieved in 5 iterations by constraining f k to be nonnegative 
and by normalizing at every step to conserve energy between 
g and 4 

Evaluation of Chromosome Enumeration Using Standard and 
EDF Imaging 

FISH probes offer a powerful means for detecting and/or 
quantifying RNA/DNA in a cell and/or cellular organelle. 
Current slide-based FISH protocols require fixation (e.g., 
with a polar organic solvent such as methanol) of intact cells. 
However, this fixation step is not compatible with in-suspen- 
sion hybridization due to the occurrence of substantial cell 
loss and cell clumping. Fluorescence In Situ Hybridization-In 
Suspension (FISH-IS) protocols for performing chromo- 
somal detection on whole cells maintained in fluid suspension 
have therefore been developed. These protocols enable the 
cells to be fixed and hybridized without significant loss of 
cells or cell clumping. FISH-IS has been successfully per- 
formed on many different cell types with a variety of probes 
that are of interest to the clinical and scientific research com- 
munities. 

Automated chromosome enumeration via FISH or FISH- 
IS probing is an application for which EDF imaging may 
confer significant benefits. Defocus causes significant 
changes in the presentation of probes often blurring one into 
another or spreading out the signal to such a degree that it is 
difficult to automatically segment, or visually separate FISH 
probes from each other or from non-specific binding in the 
nucleus. 

To compare the efficacy of chromosome enumeration 
between the standard and extended depth of field configura- 
tions, cells of the Jurkat human lymphoma line were grown in 
a suspension culture, then probed using a FISH-IS protocol. 
Cells were fixed and permeabilized with successive incuba- 
tions (5 minutes at 4° C.) in 30%, then 70% Camoy’s solution 
(3:1 methanol :acetic acid) in phosphate buffered saline 
(PBS). After centrifugation, cells were washed once in 
2xSSC (a commonly used buffer including 3 M NaCl, 0.3 M 
NaCitrate, pH 7.0), then re-suspended in a hybridization 
buffer containing a Spectrum Green labeled chromosome 8 
enumeration probe, according to the manufacturer’s direc- 
tions (V ysis). To hybridize the probe, cells in PCR tubes were 
exposed to 80° C. for 5 minutes and 42° C. for 2 hours in a 
DNA thermocycler. 1 00 ul of 2xSSC was added to the tubes, 
and the cells were pelleted by centrifugation. The pellets were 
then re-suspended in 50 ul of 1 % paraformaldehyde (in PBS). 
The sample was then loaded into the exemplary flow imaging 
system, and a file of 1 ,000 cells was collected in the standard 
collection mode (i.e., without the optical deformation ele- 
ment in place). A second file was collected from the same 
sample immediately thereafter using the WFC EDF collec- 
tionmode. Both files were analyzed in the same manner using 
IDEAS™ software to detect and enumerate chromosome 8 in 
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each cell. Image galleries were generated of cells having one, 
two, or more copies of chromosome 8. The entire collection 
time for both files was several minutes (including the time 
required to switch from standard to EDF modes). 

The results were automatically analyzed to enumerate cop- 
ies of the Y chromosome in each cell. Simple classifiers using 
brightfield imagery were developed to exclude cellular 
debris, doublet events, and other artifacts from the analysis. A 
specialized segmentation routine and connected components 
to analysis were performed on the fluorescence imagery to gen- 
erate a first pass enumeration of chromosomes on single cells. 
A refinement of the monosomy and disomy classified cells 
from the first pass enumeration was performed to eliminate 
false positive events. After the final classification step, the 
15 resulting imagery was manually reviewed to qualitatively 
judge the efficacy of the final classification. This analysis was 
not intended to be a rigorous examination of the efficacy of 
the exemplary flow imaging system with EDF for chromo- 
some enumeration. Rather, this experiment was performed to 
20 explore an application for which the exemplary flow imaging 
system with WFC EDF imaging may have a beneficial result. 

FIGS. 17A-23D present a brief overview of the analysis of 
these two files (standard images and EDF images). FIGS. 17A 
and 17B shows a sampling of 20 cells (10 in each collection 
25 mode, i.e., FIG. 17A includes 10 cells obtained using stan- 
dard imaging and FIG. 17B includes 10 cells obtained using 
WFC EDF imaging), highlighting some of the challenges in 
the classification of disomies within the Jurkat sample. The 
FISH-IS probe imagery is superimposed over a reduced con- 
30 trast brightfield image of the cells (black and white have been 
reversed to reduce the amount of black in the image, to facili- 
tate reproduction in this patent application) to provide a sense 
of scale and verify that the probes are located within the cell. 
In FIG. 17A (the cell images obtained using standard imag- 
35 ing), at least one of the FISH-IS probes is positioned out of the 
plane of focus. Consequently, the area of the out of focus 
probe increases and sometimes engulfs the second probe in 
the cell. Like the bead imagery shown in FIGS. 14 A and 14B, 
the intensity falls off significantly with defocus, making it 
40 difficult to automatically segment the probe or even see it in 
the image. Image 916 of FIG. 17A includes a cell in which 
two probes appear to be in close proximity. Slight defocus 
may have caused these relatively bright probe images to blur 
into each other, creating what appears to be a single large 
45 probe. Although it cannot be specified with certainty, this cell 
is thought to have two probes due to the total intensity of the 
probe signal and the elongated shape of the probe. In marked 
contrast, the EDF imagery presented in FIG. 17B, shows 
discrete FISH-IS spots even when they are positioned in close 
50 proximity to each other, as in images 298 and 935. Unlike the 
standard collection mode of FIG. 17A, the EDF imagery of 
FIG. 17B exhibits little to no blurring of FISH-IS spots. This 
result is also readily apparent when comparing the larger 
selection of images shown in FIGS. 22A-23D, wherein each 
55 FISH labeled chromosome appears as a bright, tightly 
focused spot. 

Development of FISH Spot Enumeration Classifier and Clas- 
sification Results 

In order to determine the efficacy of EDF imaging on the 
60 enumeration of chromosomes, an exemplary, simple five-step 
classifier was developed using the IDEAS™ analysis soft- 
ware. The first two steps involved segmentation and the selec- 
tion of appropriate objects within the data file for subsequent 
analysis (the data file includes each image collected from a 
65 sample of cells or objects run through the flow imaging sys- 
tem). Object selection was accomplished by plotting the 
brightfield aspect ratio vs . brightfield area, as shown in the dot 
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plot in FIG. 18. A gate 156 was drawn that encompassed 
primarily single cells (image 424 being exemplary of a cell in 
the gated region) and excluded cell fragments/debris (image 
845 being exemplary of a cell fragment or debris), and 
grouped cells (image 75 being exemplary of a grouping or 
cluster of cells). The gate defined a population named “Cells” 
containing 595 individual objects, to which subsequent 
analysis was applied. A similar segmentation and selection 
process was performed on the standard collection file (i.e., 
images of cells collected without using EDF imaging) and 
resulted in 588 individual objects. 

The third step in classification (graphically illustrated in 
FIGS. 19A-19C), involved refinement of the standard seg- 
mentation mask to isolate areas of local maxima in each 
fluorescence cell image. A fluorescence image 89 of a cell, 
collected in Channel 3, is shown prior to segmentation in FIG. 
19A, after initial segmentation (light blue overlay) to identify 
all areas containing light above background in FIG. 19B, and 
after morphology segmentation in FIG. 19C (light blue over- 
lay). Morphology masking is a form of contour masking to 
identify areas of local maxima contained in the initial seg- 
mentation mask. 

The fourth step in the exemplary classification employed 
an IDEAS™ feature called “FISH Spots,” which uses the 
morphology mask to perform a connected components analy- 
sis to enumerate discrete FISH spots contained within each 
fluorescent image. The results of this computation and the 
final gating of disomic cells, the fifth step in the classification, 
are shown in FIG. 20 for normal images, and in FIG. 21 for 
EDF images. The upper leftmost portion of FIGS. 20 and 21 
respectively correspond to FISH spot enumeration histo- 
grams for normal images and EDF images. The first pass 
analysis using the standard collection mode yielded enumera- 
tions of 506 monosomic, 67 disomic, and 15 polysomic (three 
or more FISH spots) cells as shown in the histogram of FIG. 
20 (the leftmost portion of the Figure). In contrast, the first 
pass enumeration with EDF imaging yielded 421 mono- 
somic, 136 disomic, and 38 polysomic cells, as shown in the 
histogram of FIG. 21 (the leftmost portion of the Figure). The 
EDF collection mode therefore produced a 2 times increase in 
the number of disomy and polysomy -classified cells, with a 
17% decrease in monosomic cells. Manual review of the 
monosomy and disomy-classified cells in both collection 
modes revealed a significant number of false classifications 
where hybridization had failed, leaving only non-specific 
binding of the FISH-IS probes within the nucleus. 

To improve classification accuracy, each population of 
monosomy and disomy -classified cells was further analyzed 
by plotting peak intensity vs. area for the fluorescence chan- 
nel. Non-specific binding generally has low peak intensity 
and large area, and therefore, plots of peak intensity vs. area 
improve discrimination of nonspecific binding events. Bi- 
variant plots of this analysis are shown in the middle portion 
of FIGS. 20 and 21, for standard images and EDF images, 
respectively. The discrimination boundaries for the standard 
collection mode are not clear. This result is most evident in a 
boundary 201 (upper center portion of FIG. 20) drawn to 
discriminate true and false positives for disomy refinement. 
The boundary is complex and arbitrary and therefore unsuit- 
able for fully automated classification. In contrast, a bound- 
ary 203 (upper center portion of FIG. 21) drawn for the EDF 
analysis is clear, with the true and false positive populations 
showing excellent separation. Minor shifts in feature values 
due to preparation differences or instrument variations will 
not significantly affect the results of the classifications, mak- 
ing these features and boundaries suitable for fully automated 
classification. The refined classifications result in 45 (the 
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table in the rightmost portion of FIG. 20) and 96 (the table in 
the rightmost portion of FIG. 21) disomy events for the stan- 
dard and EDF collection modes, respectively and a respective 
191 (the table in the center portion of FIG. 20) and 189 (the 
5 table in the center portion of FIG. 21) monosomy events. 

FIGS. 22A-22D display a random set of images from each 
set of “Refined” and “False Positive” populations defined in 
FIG. 20 (i.e., standard images). Images in FIG. 22A corre- 
spond to standard images of cells categorized as Monosomy 
to Refined in the center plot of FIG. 20. Images in FIG. 22B 
correspond to standard images of cells categorized as Mono- 
somy False Positive in the center plot of FIG. 20. Images in 
FIG. 22C correspond to standard images of cells categorized 
as Disomy Refined in the center plot of FIG. 20. Images in 
15 FIG. 22D correspond to standard images of cells categorized 
as Disomy False Positive in the center plot of FIG. 20. 

FIGS. 23A-23D display a random set of images from each 
set of “Refined” and “False Positive” populations defined in 
FIG. 21 (i.e., standard images). Images in FIG. 23A corre- 
20 spond to EDF images of cells categorized as Monosomy 
Refined in the center plot of FIG. 21. Images in FIG. 23 B 
correspond to EDF images of cells categorized as Monosomy 
False Positive in the center plot of FIG. 21. Images in FIG. 
23C correspond to EDF images of cells categorized as Dis- 
25 omy Refined in the center plot of FIG. 21 . Images in FIG. 23D 
correspond to EDF images of cells categorized as Disomy 
False Positive in the center plot of FIG. 21. 

A review of the imagery in FIGS. 22A-23D sheds light on 
why the EDF collection mode exhibits a two-fold increase in 
30 discrimination of disomic events. First, the EDF collection 
mode largely eliminated focus variation, producing small, 
bright, tightly focused spots for each hybridization event, as 
evident in both “Refined” monosomy and disomy popula- 
tions (FIGS. 23 A and 23C). Second, these tightly focused 
35 spots dramatically improved the performance of the morpho- 
logical segmentation algorithm and final classification steps 
by forming a clear demarcation between well -hybridized 
probes and non-specific binding events (the plots in the center 
and rightmost portions of FIG. 21, respectively). The false 
40 positive events shown in the galleries of FIGS. 23B and 23 D 
are a result of non-specific binding and result in large seg- 
mentation masks with low peak intensities. The use of peak 
intensity and FISH Spot area effectively discriminates false 
positive events. By contrast, it is very difficult to discriminate 
45 between non-specific binding and highly defocused probes as 
shown in FIGS. 23B and 23D. Third, by eliminating focus 
variation, probes located away from the ideal focal plane still 
appear as small spots, which is in contrast to the probe imag- 
ery found in images 213, 228, 245, 255, 257, 275, etc. shown 
50 in FIG. 22A or images 15, 251, 279, 465, and 624 etc. of FIG. 
22C. Tight focus substantially reduces the probability of 
events where a defocused probe image engulfs or contacts a 
second probe image in the cell. With standard imaging, it is a 
rarity to find imagery similar to images 55 and 247 of FIG. 
55 23C, where two probes are in close proximity and tightly 
focused. More likely than not, one of these probes will blur, 
engulfing the other, leading to a misclassification of a disomic 
event as a monosomic event. 

It is likely that future optimizations of the exemplary flow 
60 imaging system with extended depth of field will provide for 
further improvements in image quality and advanced capa- 
bilities. Since the exemplary flow imaging system collects 
imagery in a flow cuvette with image collection access to all 
four sides of the cell, unlike slide-based imaging systems, 
65 there exists the potential to develop a two-axis orthogonal 
implementation of the architecture described herein. Cou- 
pling a two-axis version of the exemplary flow imaging sys- 
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tem architecture (i.e., a flow imaging system such as that 
shown in FIG. 2) with the EDF techniques discussed above 
would provide a means to perform full 3-D cell mapping and 
optical sectioning similar to confocal techniques, but at two to 
three orders of magnitude greater speed and without photo- 
bleaching. Each axis would collect an EDF projection of the 
cell from orthogonal perspectives, enabling a 3-D reconstruc- 
tion of the cell, as is done in optical tomographic methods. 
However, unlike confocal techniques, this method would pro- 
vide an isometric perspective with consistent resolution in all 
axes. Since the cells would be mapped in a single pass, photo- 
bleaching would be minimized and, with sufficient image 
processing capacity, tens of thousands of cells could be ana- 
lyzed in several minutes. FIG. 2 schematically illustrates such 
a stereoscopic imaging system. 

The high-resolution EDF flow imaging techniques dis- 
closed herein should find beneficial application in the follow- 
ing types of image-based analytical studies: (1) FISH-IS, 
which provides high throughput automated spot counting of 
FISH-probed cells in suspension; (2) Cell Cycle and Mitosis 
Analysis for quantization and visualization of DNA-stained 
cells; (3) Stem Cell Imaging for visualization of rare cells; (4) 
Phagocytosis for quantitative analysis of macrophage activ- 
ity; and (5) Cell Signaling for imaging and quantization of 
T-cell/antigen-presenting cell conjugates. Moreover, one of 
the most promising applications is the high throughput 
genetic testing of cells using FISH-IS cell probing technique. 
Standard FISH is increasingly being used for such purposes 
as prenatal genetic testing, qualifying patients for breast can- 
cer treatment with Herceptin™, and leukemia lymphoma 
testing. Current methods of FISH probing are typically per- 
formed manually on a small number of cells per test, which 
makes them unsuitable for identifying and classifying cancer 
or other target cells that may be present at less than five 
percent of the sample. The FISH-IS technique, in connection 
with the exemplary EDF flow imaging system’s ability to 
analyze tens of thousands of cells, will allow the detection of 
rare target cells for clinical applications like cancer detection 
as well as the correlation of genetic and phenotypic traits in 
target validation studies. 

Further improvements relate to optimizing the PSF used 
for image reconstruction. More specifically, a custom model 
PSF for each channel will be created, to take into account 
different focal depths and aberrations which may be present at 
different wavelengths. Thus, different correct PSFs will be 
used for post image processing (i.e., image de-convolution). 
Such PSF s will be tailored to work with a given depth of field, 
by collecting data from beads over the full depth for which the 
system is expected to perform. 

Exemplary Computing Environment 

As discussed above, a key aspect of the EDF imaging 
techniques disclosed herein involves post image acquisition 
processing to enhance the image data, to achieve an EDF 
image. Such image processing corrects for errors introduced 
by the PSF of the imaging system, and the intentional distor- 
tion of the optical wave front from the object. Preferably, such 
image processing is a de-convolution process based on the 
PSF of the imaging system (or other corrective PSFs, gener- 
ally as discussed immediately above). FIG. 24 schematically 
illustrates an exemplary computing system 250 suitable for 
use in implementing the method of FIG. 4 (i.e., for executing 
step 170 of this method). Exemplary computing system 250 
includes a processing unit 254 that is functionally coupled to 
an input device 252 and to an output device 262, e.g., a display 
(which can be used to output a result to a user, although such 
a result can also be stored). Processing unit 254 comprises, for 
example, a central processing unit (CPU) 258 that executes 
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machine instructions for carrying out an analysis of data 
collected in connection with operation of the vehicle to deter- 
mine upon which one of the plurality of predefined routes the 
vehicle has been operated in conjunction with acquisition of 
5 the data. The machine instructions implement functions gen- 
erally consistent with those described above with respect to 
step 170 of FIG. 4, as well as those at other locations herein 
with respect to image processing to enhance the EDF image. 
CPUs suitable for this purpose are readily available, for 
10 example, from Intel Corporation, AMD Corporation, 
Motorola Corporation, and other sources, as will be well 
known to those of ordinary skill in this art. 

Also included in processing unit 254 are a random access 
15 memory (RAM) 256 and non-volatile memory 260, which 
can include read only memory (ROM) and may include some 
form of memory storage, such as a hard drive, an optical disk 
(and drive), etc. These memory devices are bi-directionally 
coupled to CPU 258. Such storage devices are well known in 
20 the art. Machine instructions and data are temporarily loaded 
into RAM 256 from non-volatile memory 260. Also stored in 
the memory are an operating system software and ancillary 
software. While not separately shown, it will be understood 
that a generally conventional power supply will be included to 
25 provide electrical power at a voltage and current level appro- 
priate to energize the components of computing system 250. 

Input device 252 can be any device or mechanism that 
facilitates user input into the operating environment, includ- 
ing, but not limited to, one or more of a mouse or other 
30 pointing device, a keyboard, a microphone, a modem, or other 
input device. In general, the input device will be used to 
initially configure computing system 250, to achieve the 
desired processing (e.g., to process image data to produce 
35 images as discussed above). Configuration of computing sys- 
tem 250 to achieve the desired processing includes the steps 
of loading appropriate processing software into non-volatile 
memory 260, and launching the processing application (e.g., 
loading the processing software into RAM 256 for execution 
40 by the CPU) so that the processing application is ready for 
use. Output device 262 generally includes any device that 
produces output information, but will most typically com- 
prise a monitor or computer display designed for human 
visual perception of output. Use of a conventional computer 
45 keyboard for input device 252 and a computer display for 
output device 262 should be considered as exemplary, rather 
than as limiting on the scope of this system. Data link 264 is 
configured to enable image data collected from a flow imag- 
ing system to be introduced into computing system 250 for 
50 subsequent image processing as discussed above. Those of 
ordinary skill in the art will readily recognize that many types 
of data links can be implemented, including, but not limited 
to, universal serial bus (USB) ports, parallel ports, serial 
ports, inputs configured to couple with portable memory stor- 
55 age devices, FireWire (conforming to I.E.E.E. 1394 specifi- 
cation) ports, infrared data ports, wireless data ports such as 
Bluetooth™, network connections such as Ethernet ports, 
and Internet connections. 

Although the concepts disclosed herein have been 
60 described in connection with the exemplary form of practic- 
ing them and modifications thereto, those of ordinary skill in 
the art will understand that many other modifications can be 
made thereto within the scope of the claims that follow. 
Accordingly, it is not intended that the scope of these con- 
65 cepts in any way be limited by the above description, but 
instead be determined entirely by reference to the claims that 
follow. 
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The invention in which an exclusive right is claimed is 
defined by the following: 

1 . A method for producing an extended depth of field image 
of an object, comprising the steps of: 

(a) deforming an optical wave front of light from the obj ect 5 

to produce modified light, such that a point spread func- 
tion (PSF) of an imaging system used to collect the 
modified light is substantially invariant across the 
extended depth of field, wherein an optical element is 
used to deform the optical wave front, the optical ele- 10 
ment comprising a cuvette having different optical 
thicknesses at different locations, such that imaging 
through the different locations of the cuvette induces 
different degrees of wave front deformation; 15 

(b) dispersing the modified light, producing dispersed 
light; 

(c) focusing the dispersed light to produce at least one 
dispersed image, the dispersed image being formed by 
light simultaneously collected from a plurality of differ- 20 
ent focal planes in the extended depth of field; 

(d) detecting the dispersed image of the object to generate 
image data; and 

(e) processing the image data to reduce artifacts introduced 
by deforming the optical wave front, to produce the 25 
extended depth of field image of the object. 

2. The method of claim 1, further comprising the step of 
analyzing the extended depth of field image of the object to 
determine at least one characteristic of the object. 

3. The method of claim 1, wherein there is relative motion 30 
between the obj ect and an imaging system used to produce the 
extended depth of field image of the object. 

4. The method of claim 3, wherein the object is entrained in 
a flow of fluid. 

5. The method of claim 1, wherein the extended depth of 35 
field comprises less than about fifteen microns. 

6. The method of claim 1, wherein the step of processing 
the image data comprises the steps of: 

(a) determining the PSF of the imaging system configured 

to deform the optical wave front; and 40 

(b) using the PSF function to produce the extended depth of 
field image of the object. 

7. The method of claim 1, wherein processing the image 
data comprises the step of de-convolving the image data. 

8. The method of claim 7, wherein the step of de-convolv- 45 
ing the image data reduces spatial broadening and contrast 
loss induced by the step of deforming the optical wave front. 

9. A method for producing an extended depth of field image 
of an object, while there is relative motion between the object 
and an imaging system used to produce the extended depth of 50 
field image of the object, comprising the steps of: 

(a) using an optical element to induce an aberration in an 
optical wave front of light from the object to produce 
modified light, such that a point spread function (PSF) of 
an imaging system as modified by the optical element is 55 
substantially invariant across the extended depth of field, 
wherein the optical element comprises a cuvette having 
different optical thicknesses at different locations, such 
that imaging through the different locations of the 
cuvette induces different degrees of wave front defor- 60 
mation; 

(b) focusing the modified light to produce an image of the 

object that includes light simultaneously collected from 
a plurality of different focal planes in the extended depth 
of field; 65 

(c) detecting the image of the object to generate image data; 
and 
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(d) processing the image data to reduce artifacts introduced 
by deforming the optical wave front, to produce the 
extended depth of field image of the object. 

10. The method of claim 9, wherein the step of detecting 
the image comprises the step of using a time delay integration 
detector to collect the image over a period of time, while the 
object moves relative to the imaging system. 

11. The method of claim 9, further comprising the step of 
dispersing the modified light into a plurality of light beams, 
such that: 

(a) the step of focusing the modified light comprises the 
step of focusing each of the light beams to produce a 
respective image corresponding to that light beam, each 
respective image being formed from the modified light 
simultaneously collected from the plurality of different 
focal planes in the extended depth of field; 

(b) the step of detecting the image comprises the step of 
detecting each respective image of the object to generate 
image data; and 

(c) the step of processing the image comprises the step of 
processing the image data for each respective image to 
reduce artifacts introduced by deforming the optical 
wave front, to produce the extended depth of field image 
of the object for each respective image. 

12. The method of claim 11, further comprising the step of 
analyzing the extended depth of field image of the object for 
each respective image to determine at least one characteristic 
of the object. 

13. The method of claim 11, wherein the step of processing 
the image data comprises the steps of: 

(a) determining the PSF of the imaging system as modified 
by the optical element configured to deform the optical 
wave front; and 

(b) using the PSF function to de-convolve each respective 
image, thereby reducing spatial broadening and contrast 
loss induced by the PSF. 

14. An imaging system adapted to perform extended depth 
of field imaging of an object, comprising: 

(a) an optical element configured to deform an optical wave 
front of light from the object, such that a point spread 
function (PSF) of the imaging system as modified by the 
optical element is substantially invariant across the 
extended depth of field, wherein the optical element is 
selectively positionable, such that in a first position, the 
optical element deforms the optical wave front, thereby 
enabling extended depth of field imaging, while in a 
second position, the optical element does not deform the 
optical wave front, thereby enabling non-extended depth 
of field imaging; 

(b) a collection lens disposed so that light traveling from 
the object passes through the collection lens and travels 
along a collection path; 

(c) a light dispersing element disposed in the collection 
path so as to disperse the light that has passed through 
the collection lens, producing dispersed light; 

(d) an imaging lens disposed to receive the dispersed light, 
producing a dispersed image from the dispersed light; 

(e) a detector disposed to receive the dispersed image pro- 
duced by the imaging lens, producing an output signal 
that is indicative of the dispersed image; and 

(f) a processor configured to manipulate the output signal 
to reduce artifacts introduced by the optical element 
configured to deform the optical wave front of light from 
the object, to produce an extended depth of field image 
of the object. 
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15. The system of claim 14, wherein the optical element 
comprises one element selected from the group consisting 
essentially of: 

(a) a phase plate configured to induce a phase deviation in 
an optical wave front; 

(b) an optical element configured to induce a spherical 
aberration in light from the object; and 

(c) a cuvette having different optical thicknesses at differ- 
ent locations, such that imaging through the different 
locations of the cuvette induces different degrees of 
wave front deformation. 

16. The system of claim 14, wherein the optical element 
configured to deform the optical wave front of light from the 
object comprises an adjustable objective lens, such that the 
aberration of the objective lens can be modified to deform the 
optical wave front of light from the object. 

17. The system of claim 14, wherein the processor is con- 
figured to use the PSF function as modified by the optical 
element to de-convolve the output signal, thereby reducing 
spatial broadening and contrast loss induced by the PSF as 
modified by the optical element. 

18. An imaging system adapted to perform extended depth 
of field imaging of an object, while there is relative movement 
between the object and the imaging system, comprising: 

(a) an optical element configured to deform an optical wave 
front of light from the object, thereby producing modi- 
fied light, such that a point spread function (PSF) of the 
imaging system as modified by the optical element is 
substantially invariant across the extended depth of field, 
wherein the optical element is selectively positionable, 
such that in a first position, the optical element deforms 
the optical wave front, thereby enabling extended depth 
of field imaging, while in a second position, the optical 
element does not deform the optical wave front, thereby 
enabling non-extended depth of field imaging; 

(b) a collection lens disposed so that modified light passes 
through the collection lens and travels along a collection 
path; 

(c) a dispersing component disposed in the collection path 
so as to receive the light that has passed through the 
collection lens, dispersing the light into a plurality of 
separate light beams, each light beam being directed 
away from the dispersing component in a different pre- 
determined direction; 

(d) an imaging lens disposed to receive the light beams 
from the dispersing component, producing a plurality of 
images, each image corresponding to one of the light 
beams and being projected by the imaging lens toward a 
different predetermined location; 


36 

(e) a detector disposed to receive the plurality of images 
produced by the imaging lens, producing an output sig- 
nal that is indicative of imaging at the plurality of dif- 
ferent focal planes; and 

(f) a processor configured to manipulate the output signal 
to reduce artifacts introduced by the optical element 
configured to deform the optical wave front of light from 
the object, to produce an extended depth of field image 
of the object for at least one of the plurality of images. 

19. The system of claim 18, wherein the optical element is 
a phase plate configured to induce a phase deviation in the 
optical wave front. 

20. The system of claim 19, wherein the phase plate is 
disposed proximate a numerical objective of the imaging 
system. 

21. The system of claim 18, wherein the optical element is 
configured to induce a spherical aberration in light from the 
object. 

22. The system of claim 18, wherein the optical element is 
a cuvette having different optical thicknesses at different 
locations, such that imaging through the different locations of 
the cuvette induces different degrees of wave front deforma- 
tion. 

23. An imaging system adapted to perform extended depth 
of field imaging of an object, comprising: 

(a) an optical element configured to deform an optical wave 
front of light from the object, such that a point spread 
function (PSF) of the imaging system as modified by the 
optical element is substantially invariant across the 
extended depth of field, wherein the optical element is a 
cuvette having different optical thicknesses at different 
locations, such that imaging through the different loca- 
tions of the cuvette induces different degrees of wave 
front deformation; 

(b) a collection lens disposed so that light traveling from 
the object passes through the collection lens and travels 
along a collection path; 

(c) a light dispersing element disposed in the collection 
path so as to disperse the light that has passed through 
the collection lens, producing dispersed light; 

(d) an imaging lens disposed to receive the dispersed light, 
producing a dispersed image from the dispersed light; 

(e) a detector disposed to receive the dispersed image pro- 
duced by the imaging lens, producing an output signal 
that is indicative of the dispersed image; and 

(f) a processor configured to manipulate the output signal 
to reduce artifacts introduced by the optical element 
configured to deform the optical wave front of light from 
the object, to produce an extended depth of field image 
of the object. 



